-- Logs begin at Mon 2022-06-27 20:28:24 UTC, end at Mon 2022-06-27 21:35:23 UTC. -- Jun 27 21:02:04.484495 dv-cinder-847733-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 27 21:02:04.492618 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 27 21:02:04.494623 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jun 27 21:02:04 2022] *** Jun 27 21:02:04.494623 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 27 21:02:04.494623 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 27 21:02:04.494623 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: nodename: dv-cinder-847733-1-iscsi Jun 27 21:02:04.494623 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: machine: x86_64 Jun 27 21:02:04.494623 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: clock source: unix Jun 27 21:02:04.494623 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: pcre jit disabled Jun 27 21:02:04.494623 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: detected number of CPU cores: 4 Jun 27 21:02:04.494623 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: current working directory: / Jun 27 21:02:04.494623 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: detected binary path: /usr/bin/uwsgi-core Jun 27 21:02:04.495504 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: your processes number limit is 47680 Jun 27 21:02:04.495504 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: your memory page size is 4096 bytes Jun 27 21:02:04.495504 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: detected max file descriptor number: 2048 Jun 27 21:02:04.495504 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: lock engine: pthread robust mutexes Jun 27 21:02:04.495504 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: thunder lock: enabled Jun 27 21:02:04.495504 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 27 21:02:04.495504 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 27 21:02:04.539889 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: Python main interpreter initialized at 0x5587ab8949c0 Jun 27 21:02:04.539889 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: python threads support enabled Jun 27 21:02:04.539889 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: your server socket listen backlog is limited to 100 connections Jun 27 21:02:04.539889 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: your mercy for graceful operations on workers is 90 seconds Jun 27 21:02:04.540310 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: mapped 403077 bytes (393 KB) for 2 cores Jun 27 21:02:04.540376 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: *** Operational MODE: preforking *** Jun 27 21:02:04.540641 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: *** uWSGI is running in multiple interpreter mode *** Jun 27 21:02:04.540641 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: spawned uWSGI master process (pid: 80387) Jun 27 21:02:04.540856 dv-cinder-847733-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 27 21:02:04.541614 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: spawned uWSGI worker 1 (pid: 80400, cores: 1) Jun 27 21:02:04.541720 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: spawned uWSGI worker 2 (pid: 80401, cores: 1) Jun 27 21:02:04.542714 dv-cinder-847733-1-iscsi devstack@keystone.service[80387]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 27 21:02:09.931829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 21:02:09.931829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: from cryptography.utils import int_from_bytes Jun 27 21:02:09.932651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 21:02:09.932651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: from cryptography.utils import int_from_bytes Jun 27 21:02:09.932762 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 21:02:09.932762 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: from cryptography.utils import int_from_bytes Jun 27 21:02:09.932762 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 21:02:09.932762 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: from cryptography.utils import int_from_bytes Jun 27 21:02:09.980792 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 27 21:02:09.981006 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 27 21:02:09.981166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] command line args: [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 27 21:02:09.981311 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 27 21:02:09.981394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 27 21:02:09.981476 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 27 21:02:09.981558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ================================================================================ {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 27 21:02:09.981706 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] command line args: [] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 27 21:02:09.981842 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] admin_token = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.981984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 27 21:02:09.982117 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] config_dir = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.982194 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ================================================================================ {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 27 21:02:09.982268 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.982449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] config_source = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.982578 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] admin_token = **** {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.982663 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] debug = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.982741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] config_dir = [] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.982819 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.983050 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.983050 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] config_source = [] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.983159 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.983222 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] debug = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.983292 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.983355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.984445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.984552 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.984552 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.984686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.984686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.984804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.984863 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_config_append = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.984925 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.984982 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.985046 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] list_limit = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.985109 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_dir = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.985171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] log_config_append = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.985235 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_file = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.985296 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.985358 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.985425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] log_dir = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.985484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.985545 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] log_file = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.985608 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.985665 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.985720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.985790 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.985847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.985905 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.985966 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.986029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.986097 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.986372 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.986499 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.986578 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.986646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.986711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.986781 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.986917 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.986993 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.987125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.987218 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.987362 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.987450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.987597 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.987704 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.987858 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.987941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.988108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.988363 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.988465 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.988594 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] publish_errors = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.988687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.988809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.989011 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.989104 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.989176 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.989304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] publish_errors = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.989403 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.989516 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.989656 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.989734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.989880 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] standard_threads = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.989966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.990105 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.990185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.990325 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.990418 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.990638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.990713 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] standard_threads = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.990779 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] use_journal = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.990837 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.990894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] use_json = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.991023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] use_stderr = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.991111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.991172 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] use_syslog = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.991317 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.991392 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] use_journal = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.991461 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.991538 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] use_json = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.992930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.993491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] use_stderr = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.994099 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.994178 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] use_syslog = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.994260 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.994332 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:02:09.994396 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.994488 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.994566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.994631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.994712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.994774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.994837 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.994918 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.995126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.995205 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.995274 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.995340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.995404 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.995479 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.995594 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.995717 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.995797 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.995869 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.995933 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.996007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.996077 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.996151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.996224 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.996294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.996294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.996417 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.996485 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.996559 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.996708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.996781 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.996845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.996913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.996987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.997052 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.997052 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.997188 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.997263 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.997332 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.997332 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.997451 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.997530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.997601 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.997676 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.997745 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.997745 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.997874 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.997950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.998020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.998125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.998217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.998382 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.998504 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.998595 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.998683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.998856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.998991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.999070 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.999154 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.999270 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.999363 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.999535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.999701 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.999799 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:09.999884 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.000000 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.000094 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.000263 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.000395 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.000475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.000559 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.000740 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.000829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.000906 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.000976 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.001151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.001355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.001437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.001508 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.001583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.001653 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.001898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.002003 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.002152 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.002274 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.002350 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.002419 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.002603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.002737 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.002818 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.002889 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.003050 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.003189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.003275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.003357 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.003499 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.003686 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.003768 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.003836 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.003980 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.004909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.004986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.005063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.005134 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.005200 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.005301 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.005383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.005456 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.005521 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.005589 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.005657 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.005782 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.005867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.005940 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.006058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.006133 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.006204 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.006320 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.006405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.006486 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.006568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.006651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.006724 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.006840 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.006916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.007043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.007160 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.007243 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.007327 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.007400 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.007514 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.007590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.007742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.007824 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.007939 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.008022 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.008095 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.008249 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.008347 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.008501 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.008624 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.008713 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.008788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.008955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.009097 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.009234 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.009322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.009401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.009473 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.009604 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.009682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.009859 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.009992 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.010073 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.010152 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.010284 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.010373 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.010543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.010667 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.010747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.010893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.010985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.011060 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.011236 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.011359 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.011448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.011523 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.011702 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.011792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.011960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.012084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.012162 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.012234 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.012392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.012468 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.012620 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.012703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.012852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.012933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.013078 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.013170 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.013306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.013389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.013529 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.013612 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.013761 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.013849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.013986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.014068 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.014223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.014305 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.014457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.014538 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.014697 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.014786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.014935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.015018 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.015171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.015252 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.015423 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.015506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.015682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.015766 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.015917 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.external = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.015997 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.016130 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.016209 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.016406 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.016483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.016638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.016723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] auth.external = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.016867 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.password = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.016954 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.017094 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.token = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.017172 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.017331 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.017409 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.017587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.017678 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] auth.password = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.017830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.017908 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] auth.token = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.018057 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.018139 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.018333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.018426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.018581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.018660 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.018809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.018887 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.019042 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.019127 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.019283 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.019367 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.019515 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.019596 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.019780 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] credential.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.019879 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.020016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.020146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.020254 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.020347 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.020487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.020574 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.020728 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.020807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.020975 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.021052 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.021207 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.021290 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.021455 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.021532 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.021714 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.021792 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.021956 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.022042 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.022215 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.022292 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.022451 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.022526 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.022706 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.022792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.022940 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.023016 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.023176 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.023253 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.023418 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.023507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.023682 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.023763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] federation.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.023915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.023992 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.024161 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.024251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.024401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.024477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.024650 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.024733 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.024885 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.024970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.025131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.025207 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.025362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.025438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.025610 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.025693 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.025843 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.025921 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.026090 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.026166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.026332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.026487 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.026562 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.026632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.026786 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.026863 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.027027 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.027111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.027254 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.027331 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.027487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.027569 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.027755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.027841 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.027989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.028065 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.028219 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.028294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.028447 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.028525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.028675 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.028751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.028918 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.028996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.029151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.029228 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.029387 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.029464 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.029636 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.029712 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.029877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.029961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.030125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.030201 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.030368 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.030444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.030603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.030680 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.030831 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.030907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.031384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.031464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.031635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.031716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.031870 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.031947 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.032112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.032189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.032349 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.032426 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.032591 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.032668 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.032816 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.032891 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.033135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.033214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.033381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.033458 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.033614 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.033690 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.033854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.033931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.034098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.034174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.034324 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.034401 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.034557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.034638 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.034781 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.034859 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.035016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.035092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.035248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.035327 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.035477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.035568 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.035820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.035922 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.036004 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.036077 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.036196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.036274 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.036440 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.036517 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.036670 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.036777 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.036898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.036978 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.037143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.037221 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.037386 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.037466 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.037620 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.037697 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.037900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.037998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.038082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.038156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.038314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.038397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.038550 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.038628 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.038791 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.038869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.039021 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.039098 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.039264 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.039341 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.039536 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.039661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.039759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.039838 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.039990 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.040082 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.040216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.040294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.040446 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.040525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.040679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.040765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.040910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.040989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.041136 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.041214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.041402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.041642 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.041722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.041875 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.041953 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.042124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.042202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.042363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.042442 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.042607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.042685 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.042834 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.042911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.043077 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.043156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.043305 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.043383 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.043536 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.043651 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.043792 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.043873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.044023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.044101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.044268 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.044345 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.044509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.044589 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.044749 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.044827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.044995 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.045072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.045220 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.045298 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.045462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.045540 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.045692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.045778 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.045922 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.046001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.046160 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.046237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.046399 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.046475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.046635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.046714 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.046863 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.046941 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.047096 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.047174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.047332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.047411 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.047571 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.047671 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.047837 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.047915 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] resource.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.048071 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.048151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.048317 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.048395 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.048554 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.048631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.048810 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.048888 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.049052 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.049130 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.049284 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.049362 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.049511 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.049586 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.049740 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] role.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.049819 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.049989 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] role.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.050074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.050217 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.050297 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] role.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.050467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.050544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] role.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.050705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.050782 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.050949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.051027 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.051192 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.051270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.051433 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.051511 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.051695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.051779 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.051942 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.052021 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.052185 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.052275 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.052417 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.052496 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.052659 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.052736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.052893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.052978 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.053135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.053212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.053376 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.053454 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.053623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.053793 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.053872 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.054028 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.054273 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.054351 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.054518 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.054603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.054758 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.054835 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.055006 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.055083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.055236 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.055326 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.055486 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.055564 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.055755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.055835 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.055988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.056079 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.056222 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.056301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.056453 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.056534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.056679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.056766 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.056917 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.056994 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.057152 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.057230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.057397 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.057475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.057630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.057709 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.057859 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.057938 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.058085 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.058165 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.058316 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.058395 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.058547 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.058625 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.058783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] token.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.058861 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.059011 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.059088 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.059260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.059336 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.059499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.059580 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.059770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.059852 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.060012 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.060091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.060252 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.060332 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.060497 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.060584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.060728 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.060870 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.060955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.061019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.061180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.061258 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.061421 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.061499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.061658 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.061736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.061883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.061959 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.062122 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.062199 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.062363 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.062441 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.062614 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.062694 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.connection = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.062892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.062978 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.063137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.063215 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.connection = **** {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.063370 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.063461 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.063649 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.063733 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.063862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.063939 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.064095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.064186 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.064461 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.064540 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.064790 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.064867 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.065033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.065120 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.065267 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.065344 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.065514 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.065591 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.065747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.065836 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.065991 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.066070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.066226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.066304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.066465 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.066551 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.066694 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.066772 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.066932 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.067009 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.067163 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.067250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.067330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 27 21:02:10.067404 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.067643 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:02:10.067821 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 27 21:02:10.098001 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.098168 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.098239 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.098300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.098440 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.098524 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.106876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.107086 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.134468 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.134668 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.135047 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.135281 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.277872 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.278409 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.326389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.326530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.326627 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.326706 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.389999 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.390151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.390244 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.390417 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.390562 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.390636 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.390712 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.390819 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.390910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.391069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.391163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.391229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.416304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.416448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.416540 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.416619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.416727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.416814 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.416994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.417143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.417214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.417291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.417398 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.417488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.417621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.417757 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.428694 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.429176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.441360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.441610 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.441748 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.441833 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.441912 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.442017 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.442104 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.442267 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.442392 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.442473 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.442547 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.442621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.442695 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.442903 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:02:10.460284 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.460495 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 21:02:10.460619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.460701 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 21:02:10.460795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.461016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 21:02:10.461141 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.461223 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 21:02:10.461339 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.461500 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.461640 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.461724 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.461797 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.461995 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.462121 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.462216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.472699 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.472972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.473095 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.473252 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.473380 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.473471 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.473555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.473791 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.481751 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.482064 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.485021 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.485302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.485426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.485580 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.485707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.485839 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.485933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.486135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.486299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.486419 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.486528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.486710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.486837 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.486978 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.487105 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.487216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.487324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.487507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.487670 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.487746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.487823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.488052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.513883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.514204 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.515471 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.515779 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.515922 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.516096 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.521041 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.521391 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.521517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.521691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.521835 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.522053 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.528976 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.529213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.529337 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.529572 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.533980 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.534264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.534392 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.534542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.534688 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.534762 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.534958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.535084 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.535154 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.535320 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.535459 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.535536 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.535730 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.535857 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.535993 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.536275 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.554665 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.554875 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.554974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.555122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.559552 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.559816 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.563159 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.563420 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.563548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.563843 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.568579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.568874 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.572161 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.572511 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.575791 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.576081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.576206 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.576366 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.576464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.576630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.576773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.576878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.576968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.577191 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.577319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.577504 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.596156 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.596435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.596564 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.596668 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.596775 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.596955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.597079 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.597299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.604811 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.605167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.606488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.606762 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.606904 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.607101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.609961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.610234 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.610357 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.610509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.610634 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.610784 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.610927 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.611031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.611249 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.611565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.624237 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.624509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.624783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.624921 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.625028 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.625118 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.625344 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.625624 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.626549 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.626857 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.627182 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.627323 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.627456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.627810 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.644875 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.645522 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.648382 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.648784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.651672 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.652103 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.653381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.653831 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.654987 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.655267 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.655481 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.655583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.655797 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.656334 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.663231 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.663756 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.664833 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.665351 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.668077 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.668361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.668555 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.668754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.668880 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.668990 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.669256 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.669535 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.678106 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.678387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.678621 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.678768 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.678874 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.679184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.686418 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.686690 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.686967 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.687091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:02:10.687237 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.687362 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.687476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.687567 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.687812 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.687935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.688044 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.688150 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.688403 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.688554 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.688662 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.688747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.688970 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.689150 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.689236 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.689301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.689522 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.689649 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.689770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.689861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.690055 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.690182 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.690335 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.690607 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:02:10.727432 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5587ab8949c0 pid: 80400 (default app) Jun 27 21:02:10.729153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5587ab8949c0 pid: 80401 (default app) Jun 27 21:02:10.730853 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd3951-27d8-4692-b961-0b0644cae849 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:10.731195 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd3951-27d8-4692-b961-0b0644cae849 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:10.731641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd3951-27d8-4692-b961-0b0644cae849 None None] PATH_INFO: `/v3` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:10.738124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 791 bytes} [Mon Jun 27 21:02:10 2022] GET /identity/v3/ => generated 255 bytes in 10 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:18.302298 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83fe2b1-793d-4686-ae8d-fa8836b1a785 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:18.302909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83fe2b1-793d-4686-ae8d-fa8836b1a785 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:18.302909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83fe2b1-793d-4686-ae8d-fa8836b1a785 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:18.307212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1/2] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:18.314235 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e19ae-a118-4db7-97d3-2caf55c3a34e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:18.314904 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e19ae-a118-4db7-97d3-2caf55c3a34e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:18.315292 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e19ae-a118-4db7-97d3-2caf55c3a34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:18.384421 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: INFO dbcounter [None req-625e19ae-a118-4db7-97d3-2caf55c3a34e None None] Registered counter for database keystone Jun 27 21:02:18.411299 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG oslo_db.sqlalchemy.engines [None req-625e19ae-a118-4db7-97d3-2caf55c3a34e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80400) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 27 21:02:18.422412 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writer thread running {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 27 21:02:18.605935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG passlib.handlers.bcrypt [None req-625e19ae-a118-4db7-97d3-2caf55c3a34e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80400) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 27 21:02:18.606275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG passlib.handlers.bcrypt [None req-625e19ae-a118-4db7-97d3-2caf55c3a34e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80400) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 27 21:02:18.661736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-625e19ae-a118-4db7-97d3-2caf55c3a34e None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:18.663463 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-625e19ae-a118-4db7-97d3-2caf55c3a34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:18.791443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 2/3] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 27 21:02:18.954368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83fe2b1-793d-4686-ae8d-fa8836b1a785 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:19.031034 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: INFO dbcounter [None req-a83fe2b1-793d-4686-ae8d-fa8836b1a785 None None] Registered counter for database keystone Jun 27 21:02:19.053003 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG oslo_db.sqlalchemy.engines [None req-a83fe2b1-793d-4686-ae8d-fa8836b1a785 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80401) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 27 21:02:19.061084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writer thread running {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 27 21:02:19.184788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83fe2b1-793d-4686-ae8d-fa8836b1a785 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:19.189785 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d3895-0ccb-4357-97cf-c9b47846746e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:19.190439 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d3895-0ccb-4357-97cf-c9b47846746e admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:19.190741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d3895-0ccb-4357-97cf-c9b47846746e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:19.191003 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d3895-0ccb-4357-97cf-c9b47846746e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:19.200158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d3895-0ccb-4357-97cf-c9b47846746e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:19.251155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1f6d3895-0ccb-4357-97cf-c9b47846746e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 27 21:02:19.251416 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d3895-0ccb-4357-97cf-c9b47846746e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:19.261902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 2/4] 10.222.0.27 () {58 vars in 1324 bytes} [Mon Jun 27 21:02:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:02:19.269002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e19ae-a118-4db7-97d3-2caf55c3a34e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:19.287851 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e19ae-a118-4db7-97d3-2caf55c3a34e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:19.296960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0271f-80e3-438a-aaa9-3167267f92f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:19.298614 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0271f-80e3-438a-aaa9-3167267f92f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:19.299198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0271f-80e3-438a-aaa9-3167267f92f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:19.299487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0271f-80e3-438a-aaa9-3167267f92f7 admin admin] PATH_INFO: `/v3/projects/2213409bed61490ba55498fcd7bd10f8` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:19.303801 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b0271f-80e3-438a-aaa9-3167267f92f7 admin admin] RBAC: Authorizing `identity:get_project(project_id=2213409bed61490ba55498fcd7bd10f8)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:19.353446 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e3b0271f-80e3-438a-aaa9-3167267f92f7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 27 21:02:19.354023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b0271f-80e3-438a-aaa9-3167267f92f7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:19.358453 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 3/5] 10.222.0.27 () {56 vars in 1207 bytes} [Mon Jun 27 21:02:19 2022] GET /identity/v3/projects/2213409bed61490ba55498fcd7bd10f8 => generated 344 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:21.360848 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45dbe01-830b-4490-8834-de32349b5425 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:21.361408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45dbe01-830b-4490-8834-de32349b5425 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:21.361408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45dbe01-830b-4490-8834-de32349b5425 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:21.365634 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 3/6] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:21.372310 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34594a6-3f91-400f-85a2-b3c50da7c51b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:21.373031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34594a6-3f91-400f-85a2-b3c50da7c51b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:21.373167 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34594a6-3f91-400f-85a2-b3c50da7c51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:21.395157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d34594a6-3f91-400f-85a2-b3c50da7c51b None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:21.396717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d34594a6-3f91-400f-85a2-b3c50da7c51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:21.407360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 4/7] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 27 21:02:21.429751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45dbe01-830b-4490-8834-de32349b5425 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:21.443528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45dbe01-830b-4490-8834-de32349b5425 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:21.448930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145da89a-f8f4-4ba6-ace5-04358b835a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:21.449565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145da89a-f8f4-4ba6-ace5-04358b835a93 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:21.449855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145da89a-f8f4-4ba6-ace5-04358b835a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:21.450127 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145da89a-f8f4-4ba6-ace5-04358b835a93 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:21.458914 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145da89a-f8f4-4ba6-ace5-04358b835a93 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:21.460755 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145da89a-f8f4-4ba6-ace5-04358b835a93 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:21.470326 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 4/8] 10.222.0.27 () {58 vars in 1324 bytes} [Mon Jun 27 21:02:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:02:21.476447 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34594a6-3f91-400f-85a2-b3c50da7c51b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:21.490832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34594a6-3f91-400f-85a2-b3c50da7c51b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:21.499765 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681489fc-d573-4c27-9538-00ce739e0de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:21.501059 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681489fc-d573-4c27-9538-00ce739e0de5 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:21.501425 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681489fc-d573-4c27-9538-00ce739e0de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:21.501779 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681489fc-d573-4c27-9538-00ce739e0de5 admin admin] PATH_INFO: `/v3/users/3dc250d3d60a40d8b744430d137fcb9c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:21.503515 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681489fc-d573-4c27-9538-00ce739e0de5 admin admin] RBAC: Authorizing `identity:get_user(user_id=3dc250d3d60a40d8b744430d137fcb9c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:21.505408 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681489fc-d573-4c27-9538-00ce739e0de5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:21.506974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 5/9] 10.222.0.27 () {56 vars in 1198 bytes} [Mon Jun 27 21:02:21 2022] GET /identity/v3/users/3dc250d3d60a40d8b744430d137fcb9c => generated 247 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:23.555361 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fe1ba-bf44-4df5-a762-133ebe33ab25 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:23.555959 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fe1ba-bf44-4df5-a762-133ebe33ab25 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:23.555959 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fe1ba-bf44-4df5-a762-133ebe33ab25 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:23.557307 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 5/10] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:23.562468 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d1f1e-8f52-438f-92ca-9b551efbb5a6 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:23.562883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d1f1e-8f52-438f-92ca-9b551efbb5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:23.563031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d1f1e-8f52-438f-92ca-9b551efbb5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:23.580794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-356d1f1e-8f52-438f-92ca-9b551efbb5a6 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:23.581983 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-356d1f1e-8f52-438f-92ca-9b551efbb5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:23.587891 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c27b5-26b3-4d98-bfb5-164e329e46e0 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:23.588183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c27b5-26b3-4d98-bfb5-164e329e46e0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:23.588441 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c27b5-26b3-4d98-bfb5-164e329e46e0 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:23.589776 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 6/11] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:23.594734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a97d89-29e5-4a0e-aa92-ad3b3b0099fd None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:23.595094 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a97d89-29e5-4a0e-aa92-ad3b3b0099fd None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:23.595383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a97d89-29e5-4a0e-aa92-ad3b3b0099fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:23.634608 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG passlib.handlers.bcrypt [None req-21a97d89-29e5-4a0e-aa92-ad3b3b0099fd None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80401) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 27 21:02:23.635028 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG passlib.handlers.bcrypt [None req-21a97d89-29e5-4a0e-aa92-ad3b3b0099fd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80401) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 27 21:02:23.671904 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 6/12] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:23.680171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-21a97d89-29e5-4a0e-aa92-ad3b3b0099fd None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:23.681771 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-21a97d89-29e5-4a0e-aa92-ad3b3b0099fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:23.695150 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d1f1e-8f52-438f-92ca-9b551efbb5a6 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:23.719906 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:23.782084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 7/13] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:23.800639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d1f1e-8f52-438f-92ca-9b551efbb5a6 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:23.803923 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a97d89-29e5-4a0e-aa92-ad3b3b0099fd None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:23.835583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bf7a4-2ced-4546-ae3d-97a58cd83f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:23.836358 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bf7a4-2ced-4546-ae3d-97a58cd83f04 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:23.836639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bf7a4-2ced-4546-ae3d-97a58cd83f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:23.836915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bf7a4-2ced-4546-ae3d-97a58cd83f04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:23.849436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bf7a4-2ced-4546-ae3d-97a58cd83f04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:23.860829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-212bf7a4-2ced-4546-ae3d-97a58cd83f04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:23.860829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:23.860829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:23.860829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:23.860829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:23.860829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:23.860829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:23.860829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:23.860829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:23.860829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:23.860829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:23.860829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:23.860829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:23.860829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:23.860829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:23.860829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:23.861977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:23.861977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:23.861977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:23.861977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:23.861977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:23.861977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:23.861977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:23.861977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:23.861977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:23.861977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:23.861977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:23.861977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:23.861977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:23.861977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:23.861977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:23.861977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:23.862807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:23.862807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:23.862807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:23.862807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:23.862807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:23.862807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:23.862807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:23.862807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:23.862807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:23.862807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:23.862807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:23.862807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:02:23.862807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 7/14] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:02:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:23.869646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bf7a4-2ced-4546-ae3d-97a58cd83f04 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:23.902944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a97d89-29e5-4a0e-aa92-ad3b3b0099fd None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:23.905268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:23.941729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4f3e0-7700-47ee-9162-103639606a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:23.942631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4f3e0-7700-47ee-9162-103639606a0d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:23.942921 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4f3e0-7700-47ee-9162-103639606a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:23.943199 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4f3e0-7700-47ee-9162-103639606a0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:23.952300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:23.971000 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bf7a4-2ced-4546-ae3d-97a58cd83f04 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:24.005622 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3bb1b6-a6f7-4989-a741-c456d6f6b889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:24.006397 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3bb1b6-a6f7-4989-a741-c456d6f6b889 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:24.006685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3bb1b6-a6f7-4989-a741-c456d6f6b889 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:24.006965 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3bb1b6-a6f7-4989-a741-c456d6f6b889 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:24.007777 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3bb1b6-a6f7-4989-a741-c456d6f6b889 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:24.008068 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3bb1b6-a6f7-4989-a741-c456d6f6b889 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:24.009482 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3bb1b6-a6f7-4989-a741-c456d6f6b889 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:24.013913 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4f3e0-7700-47ee-9162-103639606a0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:24.015463 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4f3e0-7700-47ee-9162-103639606a0d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:24.021170 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 8/15] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:24.026720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3bb1b6-a6f7-4989-a741-c456d6f6b889 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:24.116518 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 8/16] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 21:02:24.122162 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4f3e0-7700-47ee-9162-103639606a0d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:24.125055 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:24.126721 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3bb1b6-a6f7-4989-a741-c456d6f6b889 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:24.160702 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892fd56-2db7-44f0-9995-d0d70fca11a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:24.161589 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892fd56-2db7-44f0-9995-d0d70fca11a5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:24.161870 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892fd56-2db7-44f0-9995-d0d70fca11a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:24.162142 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892fd56-2db7-44f0-9995-d0d70fca11a5 None admin] PATH_INFO: `/v3/users/3dc250d3d60a40d8b744430d137fcb9c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:24.163337 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892fd56-2db7-44f0-9995-d0d70fca11a5 None admin] RBAC: Authorizing `identity:get_user(user_id=3dc250d3d60a40d8b744430d137fcb9c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:24.164855 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892fd56-2db7-44f0-9995-d0d70fca11a5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:24.166724 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 9/17] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:24 2022] GET /identity/v3/users/3dc250d3d60a40d8b744430d137fcb9c => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:24.171916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892fd56-2db7-44f0-9995-d0d70fca11a5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:24.185639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:24.223225 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4f3e0-7700-47ee-9162-103639606a0d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:24.257866 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37feeca-a7a0-4339-9929-840b448f5045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:24.258665 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37feeca-a7a0-4339-9929-840b448f5045 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:24.258945 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37feeca-a7a0-4339-9929-840b448f5045 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:24.259217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37feeca-a7a0-4339-9929-840b448f5045 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:24.268770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892fd56-2db7-44f0-9995-d0d70fca11a5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:24.279042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37feeca-a7a0-4339-9929-840b448f5045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:24.280573 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37feeca-a7a0-4339-9929-840b448f5045 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:24.302799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-d37feeca-a7a0-4339-9929-840b448f5045 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:02:24.302799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:24.302799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:24.302799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:24.302799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:24.302799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:24.302799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:24.302799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:24.302799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:24.302799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:24.302799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:24.302799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:24.302799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:02:24.302799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:02:24.302799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:02:24.303694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:02:24.303694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:24.303694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:24.303694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:24.303694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:24.303694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:24.303694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:24.303694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:24.303694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:24.303694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:24.303694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:24.303694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:24.303694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:24.303694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:24.303694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:24.303694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:24.303694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:24.304478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:02:24.304478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:02:24.304478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:02:24.304478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:02:24.304478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 9/18] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:24.304738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f5993-447a-4418-876e-e13d6033eb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:24.304943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f5993-447a-4418-876e-e13d6033eb42 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:24.305197 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f5993-447a-4418-876e-e13d6033eb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:24.305493 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f5993-447a-4418-876e-e13d6033eb42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:24.306694 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f5993-447a-4418-876e-e13d6033eb42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:24.308189 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f5993-447a-4418-876e-e13d6033eb42 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:24.310012 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 10/19] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:24.310128 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37feeca-a7a0-4339-9929-840b448f5045 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:24.316043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f5993-447a-4418-876e-e13d6033eb42 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:24.367005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:24.395643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:24.410881 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37feeca-a7a0-4339-9929-840b448f5045 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:24.416315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f5993-447a-4418-876e-e13d6033eb42 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:24.442969 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975f72d-8697-4d8f-a611-075158028686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:24.443675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975f72d-8697-4d8f-a611-075158028686 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:24.443962 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975f72d-8697-4d8f-a611-075158028686 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:24.444239 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975f72d-8697-4d8f-a611-075158028686 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:24.444969 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975f72d-8697-4d8f-a611-075158028686 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:24.445279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975f72d-8697-4d8f-a611-075158028686 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:24.446693 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975f72d-8697-4d8f-a611-075158028686 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:24.450228 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a80c3-d84c-4c8f-892f-ccd7525db140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:24.450895 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a80c3-d84c-4c8f-892f-ccd7525db140 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:24.451180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a80c3-d84c-4c8f-892f-ccd7525db140 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:24.451454 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a80c3-d84c-4c8f-892f-ccd7525db140 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:24.452389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a80c3-d84c-4c8f-892f-ccd7525db140 None admin] RBAC: Adding query filter params (role.id=5c3c095bb53943eca95c4e049cd8cbf5, scope.domain.id=default, user.id=3dc250d3d60a40d8b744430d137fcb9c) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:24.452700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a80c3-d84c-4c8f-892f-ccd7525db140 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:24.454140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a80c3-d84c-4c8f-892f-ccd7525db140 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:24.461076 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 11/20] 10.222.0.27 () {56 vars in 1322 bytes} [Mon Jun 27 21:02:24 2022] GET /identity/v3/role_assignments?user.id=3dc250d3d60a40d8b744430d137fcb9c&scope.domain.id=default&role.id=5c3c095bb53943eca95c4e049cd8cbf5 => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:24.464315 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 10/21] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:26.794673 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1140b-e30c-44fe-9c7a-8f564866dba1 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:26.795716 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1140b-e30c-44fe-9c7a-8f564866dba1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:26.796221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1140b-e30c-44fe-9c7a-8f564866dba1 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:26.797822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 12/22] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:26.803514 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8171400-a71c-4ba3-9547-9b5fe042689b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:26.803970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8171400-a71c-4ba3-9547-9b5fe042689b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:26.804159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8171400-a71c-4ba3-9547-9b5fe042689b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:26.829309 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b8171400-a71c-4ba3-9547-9b5fe042689b None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:26.830601 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b8171400-a71c-4ba3-9547-9b5fe042689b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:26.926322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 11/23] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:26.960915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1140b-e30c-44fe-9c7a-8f564866dba1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:26.987359 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa47bc5-f92e-4610-9c4f-993ab414d54b None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:26.987693 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa47bc5-f92e-4610-9c4f-993ab414d54b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:26.988222 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa47bc5-f92e-4610-9c4f-993ab414d54b None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:26.989966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 12/24] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:26.995388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacc49b-3021-481d-952e-dcbcc7e8df09 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:26.995905 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacc49b-3021-481d-952e-dcbcc7e8df09 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:26.996399 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacc49b-3021-481d-952e-dcbcc7e8df09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:27.032934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-7dacc49b-3021-481d-952e-dcbcc7e8df09 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:27.034688 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7dacc49b-3021-481d-952e-dcbcc7e8df09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:27.114900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1140b-e30c-44fe-9c7a-8f564866dba1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:27.133676 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:27.150862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a74d8-3c72-499f-a431-26f4e731a72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:27.151936 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a74d8-3c72-499f-a431-26f4e731a72e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:27.152371 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a74d8-3c72-499f-a431-26f4e731a72e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:27.152819 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a74d8-3c72-499f-a431-26f4e731a72e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:27.164832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a74d8-3c72-499f-a431-26f4e731a72e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:27.166674 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a74d8-3c72-499f-a431-26f4e731a72e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:27.169963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:27.190717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-e60a74d8-3c72-499f-a431-26f4e731a72e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:02:27.190717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:27.190717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:27.190717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:27.190717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:27.190717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:27.190717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:27.190717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:27.190717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:27.190717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:27.190717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:27.190717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:27.190717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:02:27.190717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:02:27.190717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:02:27.191575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:02:27.191575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:27.191575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:27.191575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:27.191575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:27.191575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:27.191575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:27.191575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:27.191575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:27.191575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:27.191575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:27.191575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:27.191575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:27.191575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:27.191575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:27.191575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:27.191575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:27.192422 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:02:27.192422 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:02:27.192422 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:02:27.192422 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:02:27.194379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 13/25] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:27.194693 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 13/26] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:27.196383 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cbb7f-0866-4da4-af8b-70d0cf4fbd5a None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:27.196882 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cbb7f-0866-4da4-af8b-70d0cf4fbd5a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:27.197345 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cbb7f-0866-4da4-af8b-70d0cf4fbd5a None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:27.198826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 14/27] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:27.200752 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacc49b-3021-481d-952e-dcbcc7e8df09 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:27.205125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50d1cc-bae9-4764-87be-bae909360891 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:27.206528 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50d1cc-bae9-4764-87be-bae909360891 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:27.207040 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50d1cc-bae9-4764-87be-bae909360891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:27.236393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-dd50d1cc-bae9-4764-87be-bae909360891 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:27.241942 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-dd50d1cc-bae9-4764-87be-bae909360891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:27.341883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacc49b-3021-481d-952e-dcbcc7e8df09 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:27.349162 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 15/28] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:27.350960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50d1cc-bae9-4764-87be-bae909360891 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:27.390601 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15118cd-28f1-4c00-b6dd-d7f5563efa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:27.391747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15118cd-28f1-4c00-b6dd-d7f5563efa56 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:27.392205 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15118cd-28f1-4c00-b6dd-d7f5563efa56 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:27.392645 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15118cd-28f1-4c00-b6dd-d7f5563efa56 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:27.393608 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15118cd-28f1-4c00-b6dd-d7f5563efa56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:27.394080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15118cd-28f1-4c00-b6dd-d7f5563efa56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:27.395886 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15118cd-28f1-4c00-b6dd-d7f5563efa56 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:27.409804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 14/29] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:27.411802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22d63c-00dc-44c9-81ed-ebc216687e16 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:27.412263 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22d63c-00dc-44c9-81ed-ebc216687e16 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:27.412690 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22d63c-00dc-44c9-81ed-ebc216687e16 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:27.414257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 15/30] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:27.416000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22d63c-00dc-44c9-81ed-ebc216687e16 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:27.456274 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:27.471812 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50d1cc-bae9-4764-87be-bae909360891 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:27.498712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:27.508488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb2a2c6-3f21-4ce2-8224-231d331643a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:27.509370 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2a2c6-3f21-4ce2-8224-231d331643a8 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:27.509792 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2a2c6-3f21-4ce2-8224-231d331643a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:27.510215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2a2c6-3f21-4ce2-8224-231d331643a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:27.511200 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2a2c6-3f21-4ce2-8224-231d331643a8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:27.512914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2a2c6-3f21-4ce2-8224-231d331643a8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:27.529476 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22d63c-00dc-44c9-81ed-ebc216687e16 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:27.564202 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebf9b7-a41c-4b7a-b34c-bc05da4de43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:27.565177 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebf9b7-a41c-4b7a-b34c-bc05da4de43e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:27.565555 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebf9b7-a41c-4b7a-b34c-bc05da4de43e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:27.565924 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebf9b7-a41c-4b7a-b34c-bc05da4de43e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:27.579172 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebf9b7-a41c-4b7a-b34c-bc05da4de43e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:27.588343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-a8ebf9b7-a41c-4b7a-b34c-bc05da4de43e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:27.588343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:27.588343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:27.588343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:27.588343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:27.588343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:27.588343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:27.588343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:27.588343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:27.588343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:27.588343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:27.588343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:27.588343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:27.588343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:27.588343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:27.588343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:27.589251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:27.589251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:27.589251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:27.589251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:27.589251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:27.589251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:27.589251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:27.589251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:27.589251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:27.589251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:27.589251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:27.589251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:27.589251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:27.589251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:27.589251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:27.589251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:27.589990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:27.589990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:27.589990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:27.589990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:27.589990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:27.589990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:27.589990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:27.589990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:27.589990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:27.589990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:27.589990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:27.589990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:02:27.597310 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 16/31] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:02:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:27.598347 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebf9b7-a41c-4b7a-b34c-bc05da4de43e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:27.646346 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 16/32] 10.222.0.27 () {60 vars in 1131 bytes} [Mon Jun 27 21:02:27 2022] POST /identity/v3/roles => generated 225 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:02:27.647934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a895d7-837e-46d7-b566-4a8c40167798 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:27.648232 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a895d7-837e-46d7-b566-4a8c40167798 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:27.648514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a895d7-837e-46d7-b566-4a8c40167798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:27.668227 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-72a895d7-837e-46d7-b566-4a8c40167798 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:27.669866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-72a895d7-837e-46d7-b566-4a8c40167798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:27.700467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:27.707748 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebf9b7-a41c-4b7a-b34c-bc05da4de43e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:27.745862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce9473-6f1c-4da9-8334-6ef14dec363e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:27.746962 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce9473-6f1c-4da9-8334-6ef14dec363e None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:27.747335 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 17/33] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:27.747475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce9473-6f1c-4da9-8334-6ef14dec363e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:27.747962 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce9473-6f1c-4da9-8334-6ef14dec363e None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:27.748724 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a895d7-837e-46d7-b566-4a8c40167798 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:27.749217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ce9473-6f1c-4da9-8334-6ef14dec363e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:27.751026 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ce9473-6f1c-4da9-8334-6ef14dec363e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:27.860072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a895d7-837e-46d7-b566-4a8c40167798 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:27.878873 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:27.899343 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8575e-cdb3-4c22-b8ac-dc254e0245b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:27.900339 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8575e-cdb3-4c22-b8ac-dc254e0245b9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:27.900805 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8575e-cdb3-4c22-b8ac-dc254e0245b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:27.901247 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8575e-cdb3-4c22-b8ac-dc254e0245b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:27.902242 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8575e-cdb3-4c22-b8ac-dc254e0245b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:27.902740 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8575e-cdb3-4c22-b8ac-dc254e0245b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:27.904539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8575e-cdb3-4c22-b8ac-dc254e0245b9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:27.914993 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 18/34] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:27.916139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8575e-cdb3-4c22-b8ac-dc254e0245b9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:27.976491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 17/35] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Jun 27 21:02:27 2022] POST /identity/v3/projects => generated 301 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:02:27.984280 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce9473-6f1c-4da9-8334-6ef14dec363e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:28.079009 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8575e-cdb3-4c22-b8ac-dc254e0245b9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:28.153281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23658079-be18-4d36-b0b3-e665854005f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:28.153904 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23658079-be18-4d36-b0b3-e665854005f2 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:28.154185 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23658079-be18-4d36-b0b3-e665854005f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:28.154460 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23658079-be18-4d36-b0b3-e665854005f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:28.155220 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23658079-be18-4d36-b0b3-e665854005f2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:28.156688 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23658079-be18-4d36-b0b3-e665854005f2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:28.182063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce9473-6f1c-4da9-8334-6ef14dec363e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:28.191193 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 19/36] 10.222.0.27 () {60 vars in 1131 bytes} [Mon Jun 27 21:02:27 2022] POST /identity/v3/roles => generated 219 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:02:28.224377 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737571f9-f160-45d6-84b3-67ee6c99dc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:28.225488 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737571f9-f160-45d6-84b3-67ee6c99dc44 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:28.225773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737571f9-f160-45d6-84b3-67ee6c99dc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:28.226055 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737571f9-f160-45d6-84b3-67ee6c99dc44 None admin] PATH_INFO: `/v3/users/3dc250d3d60a40d8b744430d137fcb9c` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:28.231660 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737571f9-f160-45d6-84b3-67ee6c99dc44 None admin] RBAC: Authorizing `identity:get_user(user_id=3dc250d3d60a40d8b744430d137fcb9c)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:28.231660 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737571f9-f160-45d6-84b3-67ee6c99dc44 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:28.232444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 18/37] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:27 2022] GET /identity/v3/users/3dc250d3d60a40d8b744430d137fcb9c => generated 247 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:28.238073 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23658079-be18-4d36-b0b3-e665854005f2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:28.300214 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:28.363733 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23658079-be18-4d36-b0b3-e665854005f2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:28.401536 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6c41f-e35c-42e0-8475-e72cb6a8d51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:28.402414 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6c41f-e35c-42e0-8475-e72cb6a8d51e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:28.402960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6c41f-e35c-42e0-8475-e72cb6a8d51e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:28.403240 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6c41f-e35c-42e0-8475-e72cb6a8d51e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:28.404575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6c41f-e35c-42e0-8475-e72cb6a8d51e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:28.406023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6c41f-e35c-42e0-8475-e72cb6a8d51e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:28.408409 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 20/38] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:28.415072 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737571f9-f160-45d6-84b3-67ee6c99dc44 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:28.438782 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:28.452319 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5191ad1-e2d9-423d-b3c2-a56dd9a5c1a3 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:28.452634 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5191ad1-e2d9-423d-b3c2-a56dd9a5c1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:28.452892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5191ad1-e2d9-423d-b3c2-a56dd9a5c1a3 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:28.454366 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 21/39] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:28.457968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ce281-566a-4688-9c9c-027658395611 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:28.458258 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ce281-566a-4688-9c9c-027658395611 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:28.458544 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ce281-566a-4688-9c9c-027658395611 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:28.459873 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 22/40] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:28.460997 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed0f65-2eb9-4bfb-8715-95ec6f0facca None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:28.461289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed0f65-2eb9-4bfb-8715-95ec6f0facca None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:28.461572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed0f65-2eb9-4bfb-8715-95ec6f0facca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:28.480995 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-30ed0f65-2eb9-4bfb-8715-95ec6f0facca None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:28.482203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-30ed0f65-2eb9-4bfb-8715-95ec6f0facca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:28.532961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737571f9-f160-45d6-84b3-67ee6c99dc44 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:28.551656 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 23/41] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:28.551988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9a6ec-dcdc-49c8-9b78-c6cc398f311f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:28.552281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9a6ec-dcdc-49c8-9b78-c6cc398f311f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:28.552566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9a6ec-dcdc-49c8-9b78-c6cc398f311f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:28.570624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a0a9a6ec-dcdc-49c8-9b78-c6cc398f311f None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:28.570811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82374a2-2702-4d1a-9eec-e854336cda94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:28.571517 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82374a2-2702-4d1a-9eec-e854336cda94 None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:28.571831 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82374a2-2702-4d1a-9eec-e854336cda94 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:28.571921 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a0a9a6ec-dcdc-49c8-9b78-c6cc398f311f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:28.572082 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82374a2-2702-4d1a-9eec-e854336cda94 None admin] PATH_INFO: `/v3/domains/default/users/3dc250d3d60a40d8b744430d137fcb9c/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:28.574349 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82374a2-2702-4d1a-9eec-e854336cda94 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3dc250d3d60a40d8b744430d137fcb9c, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:28.575857 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82374a2-2702-4d1a-9eec-e854336cda94 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:28.592505 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 19/42] 10.222.0.27 () {58 vars in 1361 bytes} [Mon Jun 27 21:02:28 2022] PUT /identity/v3/domains/default/users/3dc250d3d60a40d8b744430d137fcb9c/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:02:28.593310 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82374a2-2702-4d1a-9eec-e854336cda94 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:28.602852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:28.700724 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:28.730590 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82374a2-2702-4d1a-9eec-e854336cda94 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:28.765840 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 24/43] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:28.774900 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c48056-22c6-4bb5-a797-075c0cf9a7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:28.775596 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c48056-22c6-4bb5-a797-075c0cf9a7b9 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:28.775961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c48056-22c6-4bb5-a797-075c0cf9a7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:28.776237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c48056-22c6-4bb5-a797-075c0cf9a7b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:28.776998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c48056-22c6-4bb5-a797-075c0cf9a7b9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:28.778404 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c48056-22c6-4bb5-a797-075c0cf9a7b9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:28.787798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9a6ec-dcdc-49c8-9b78-c6cc398f311f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:28.802292 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 20/44] 10.222.0.27 () {60 vars in 1131 bytes} [Mon Jun 27 21:02:28 2022] POST /identity/v3/roles => generated 223 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:02:28.901329 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9a6ec-dcdc-49c8-9b78-c6cc398f311f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:28.937938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62ae6f-ed31-4261-83ad-415d018c6c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:28.938762 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62ae6f-ed31-4261-83ad-415d018c6c4f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:28.939045 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62ae6f-ed31-4261-83ad-415d018c6c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:28.939321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62ae6f-ed31-4261-83ad-415d018c6c4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:28.940718 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62ae6f-ed31-4261-83ad-415d018c6c4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:28.942202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62ae6f-ed31-4261-83ad-415d018c6c4f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:28.943912 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 25/45] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:28.949738 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c48056-22c6-4bb5-a797-075c0cf9a7b9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:29.068897 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c48056-22c6-4bb5-a797-075c0cf9a7b9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:29.102622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f20ca7-0a7d-4924-8a1d-59bf69643710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:29.103294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f20ca7-0a7d-4924-8a1d-59bf69643710 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:29.103579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f20ca7-0a7d-4924-8a1d-59bf69643710 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:29.103894 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f20ca7-0a7d-4924-8a1d-59bf69643710 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:29.104663 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f20ca7-0a7d-4924-8a1d-59bf69643710 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:29.106172 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f20ca7-0a7d-4924-8a1d-59bf69643710 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:29.110601 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:29.134953 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 21/46] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Jun 27 21:02:28 2022] POST /identity/v3/projects => generated 312 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:02:30.605999 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9ab38-5f97-4221-985b-ea6a69ac8e3a None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:30.606523 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9ab38-5f97-4221-985b-ea6a69ac8e3a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:30.606603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9ab38-5f97-4221-985b-ea6a69ac8e3a None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:30.608091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 26/47] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:30.613821 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1baf301-a152-4fab-9ec9-a8c47daf08b4 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:30.614187 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1baf301-a152-4fab-9ec9-a8c47daf08b4 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:30.614460 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1baf301-a152-4fab-9ec9-a8c47daf08b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:30.632540 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b1baf301-a152-4fab-9ec9-a8c47daf08b4 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:30.634091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b1baf301-a152-4fab-9ec9-a8c47daf08b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:30.705545 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 22/48] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:30.728778 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9ab38-5f97-4221-985b-ea6a69ac8e3a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:30.766763 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:30.837873 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9ab38-5f97-4221-985b-ea6a69ac8e3a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:30.872215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb728d0-d7a2-4acd-9f71-d69e4416ed8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:30.872940 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb728d0-d7a2-4acd-9f71-d69e4416ed8f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:30.873223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb728d0-d7a2-4acd-9f71-d69e4416ed8f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:30.873495 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb728d0-d7a2-4acd-9f71-d69e4416ed8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:30.884332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb728d0-d7a2-4acd-9f71-d69e4416ed8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:30.892039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-6fb728d0-d7a2-4acd-9f71-d69e4416ed8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:30.892039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:30.892039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:30.892039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:30.892039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:30.892039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:30.892039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:30.892039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:30.892039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:30.892039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:30.892039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:30.892039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:30.892039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:30.892039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:30.892039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:30.892039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:30.892869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:30.892869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:30.892869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:30.892869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:30.892869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:30.892869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:30.892869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:30.892869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:30.892869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:30.892869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:30.892869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:30.892869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:30.892869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:30.892869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:30.892869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:30.892869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:30.893680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:30.893680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:30.893680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:30.893680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:30.893680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:30.893680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:30.893680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:30.893680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:30.893680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:30.893680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:30.893680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:30.893680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:02:30.893680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 27/49] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:02:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:30.899252 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1baf301-a152-4fab-9ec9-a8c47daf08b4 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:31.004081 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1baf301-a152-4fab-9ec9-a8c47daf08b4 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:31.022434 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:31.041944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07dd96-ccb6-46d5-b35d-950b3349692a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:31.042638 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07dd96-ccb6-46d5-b35d-950b3349692a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:31.042922 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07dd96-ccb6-46d5-b35d-950b3349692a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:31.043204 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07dd96-ccb6-46d5-b35d-950b3349692a None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:31.044005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de07dd96-ccb6-46d5-b35d-950b3349692a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:31.044318 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de07dd96-ccb6-46d5-b35d-950b3349692a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:31.045747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de07dd96-ccb6-46d5-b35d-950b3349692a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:31.062646 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 23/50] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:31.063938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb728d0-d7a2-4acd-9f71-d69e4416ed8f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:31.114027 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10121e2-15d5-4602-829a-8f1b94db4c44 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:31.114309 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10121e2-15d5-4602-829a-8f1b94db4c44 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:31.114588 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10121e2-15d5-4602-829a-8f1b94db4c44 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:31.115998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 24/51] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:31.120931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913cf67-f24c-426b-80dc-36b089b5713c None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:31.121226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913cf67-f24c-426b-80dc-36b089b5713c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:31.121487 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913cf67-f24c-426b-80dc-36b089b5713c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:31.138880 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a913cf67-f24c-426b-80dc-36b089b5713c None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:31.140061 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a913cf67-f24c-426b-80dc-36b089b5713c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:31.162683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:31.170584 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb728d0-d7a2-4acd-9f71-d69e4416ed8f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:31.203767 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2d6eb-2224-47c9-adcf-ba8d081291b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:31.204661 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2d6eb-2224-47c9-adcf-ba8d081291b2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:31.204946 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2d6eb-2224-47c9-adcf-ba8d081291b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:31.205224 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2d6eb-2224-47c9-adcf-ba8d081291b2 None admin] PATH_INFO: `/v3/users/3dc250d3d60a40d8b744430d137fcb9c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:31.206427 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2d6eb-2224-47c9-adcf-ba8d081291b2 None admin] RBAC: Authorizing `identity:get_user(user_id=3dc250d3d60a40d8b744430d137fcb9c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:31.207924 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2d6eb-2224-47c9-adcf-ba8d081291b2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:31.209611 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 28/52] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:31 2022] GET /identity/v3/users/3dc250d3d60a40d8b744430d137fcb9c => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:31.210870 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 25/53] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:31.214932 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2d6eb-2224-47c9-adcf-ba8d081291b2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:31.233643 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913cf67-f24c-426b-80dc-36b089b5713c None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:31.315553 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2d6eb-2224-47c9-adcf-ba8d081291b2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:31.331267 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913cf67-f24c-426b-80dc-36b089b5713c None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:31.353866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b62fb4-8db8-47bb-9404-00cad08c8add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:31.354696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b62fb4-8db8-47bb-9404-00cad08c8add None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:31.354978 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b62fb4-8db8-47bb-9404-00cad08c8add None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:31.355256 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b62fb4-8db8-47bb-9404-00cad08c8add None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:31.356562 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b62fb4-8db8-47bb-9404-00cad08c8add None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:31.358094 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b62fb4-8db8-47bb-9404-00cad08c8add None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:31.360074 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 29/54] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:31.365979 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b62fb4-8db8-47bb-9404-00cad08c8add None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:31.400584 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:31.449372 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:31.474198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec79207-4969-4a7b-944c-c0171d64ed0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:31.474994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec79207-4969-4a7b-944c-c0171d64ed0b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:31.475276 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec79207-4969-4a7b-944c-c0171d64ed0b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:31.475365 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b62fb4-8db8-47bb-9404-00cad08c8add None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:31.475549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec79207-4969-4a7b-944c-c0171d64ed0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:31.476738 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec79207-4969-4a7b-944c-c0171d64ed0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:31.478196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec79207-4969-4a7b-944c-c0171d64ed0b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:31.479843 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 26/55] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:31.485446 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec79207-4969-4a7b-944c-c0171d64ed0b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:31.507876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53a661-12d6-4099-97a9-43fd766805d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:31.508546 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53a661-12d6-4099-97a9-43fd766805d9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:31.508841 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53a661-12d6-4099-97a9-43fd766805d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:31.509113 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53a661-12d6-4099-97a9-43fd766805d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:31.509946 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53a661-12d6-4099-97a9-43fd766805d9 None admin] RBAC: Adding query filter params (role.id=5c3c095bb53943eca95c4e049cd8cbf5, scope.domain.id=default, user.id=3dc250d3d60a40d8b744430d137fcb9c) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:31.510258 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53a661-12d6-4099-97a9-43fd766805d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:31.511746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53a661-12d6-4099-97a9-43fd766805d9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:31.517648 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 30/56] 10.222.0.27 () {56 vars in 1322 bytes} [Mon Jun 27 21:02:31 2022] GET /identity/v3/role_assignments?user.id=3dc250d3d60a40d8b744430d137fcb9c&scope.domain.id=default&role.id=5c3c095bb53943eca95c4e049cd8cbf5 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:31.598928 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec79207-4969-4a7b-944c-c0171d64ed0b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:31.636347 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f53eea-1736-4599-9821-a8b627da32b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:31.637049 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f53eea-1736-4599-9821-a8b627da32b5 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:31.637284 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f53eea-1736-4599-9821-a8b627da32b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:31.637577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f53eea-1736-4599-9821-a8b627da32b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:31.638310 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f53eea-1736-4599-9821-a8b627da32b5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:31.639821 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f53eea-1736-4599-9821-a8b627da32b5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:31.670891 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 27/57] 10.222.0.27 () {60 vars in 1140 bytes} [Mon Jun 27 21:02:31 2022] POST /identity/v3/projects => generated 298 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:02:33.671997 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac666332-916d-4d9c-a746-b789b453ec36 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:33.672824 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac666332-916d-4d9c-a746-b789b453ec36 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:33.672824 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac666332-916d-4d9c-a746-b789b453ec36 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:33.674870 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 31/58] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:33.680191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf72c14-1923-49bd-8202-b21a07ac0313 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:33.680603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf72c14-1923-49bd-8202-b21a07ac0313 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:33.680759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf72c14-1923-49bd-8202-b21a07ac0313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:33.697684 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6cf72c14-1923-49bd-8202-b21a07ac0313 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:33.698895 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6cf72c14-1923-49bd-8202-b21a07ac0313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:33.723085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:33.772173 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 28/59] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:33.797211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac666332-916d-4d9c-a746-b789b453ec36 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:33.896656 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:33.909572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac666332-916d-4d9c-a746-b789b453ec36 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:33.945609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290cdc6-b0d4-4b5f-bd41-f2ddd847fef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:33.946365 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290cdc6-b0d4-4b5f-bd41-f2ddd847fef8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:33.946648 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290cdc6-b0d4-4b5f-bd41-f2ddd847fef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:33.946928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290cdc6-b0d4-4b5f-bd41-f2ddd847fef8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:33.948134 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290cdc6-b0d4-4b5f-bd41-f2ddd847fef8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:33.949611 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290cdc6-b0d4-4b5f-bd41-f2ddd847fef8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:33.951223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 32/60] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:33.959419 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf72c14-1923-49bd-8202-b21a07ac0313 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:34.067157 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf72c14-1923-49bd-8202-b21a07ac0313 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:34.103379 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a510e86-31ea-48f1-8247-6d96ae447e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:34.104098 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a510e86-31ea-48f1-8247-6d96ae447e3e None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:34.104413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a510e86-31ea-48f1-8247-6d96ae447e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:34.104769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a510e86-31ea-48f1-8247-6d96ae447e3e None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:34.105874 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a510e86-31ea-48f1-8247-6d96ae447e3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:34.107939 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a510e86-31ea-48f1-8247-6d96ae447e3e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:34.144204 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 29/61] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Jun 27 21:02:33 2022] POST /identity/v3/users => generated 275 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:02:37.365910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cedb1-40ea-426f-b940-80fabc467d8f None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.366458 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cedb1-40ea-426f-b940-80fabc467d8f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.366458 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cedb1-40ea-426f-b940-80fabc467d8f None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.367867 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 33/62] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:37.383083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437a1fb-6341-44d3-ae8d-752f0b7dc546 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.383908 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437a1fb-6341-44d3-ae8d-752f0b7dc546 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.384348 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437a1fb-6341-44d3-ae8d-752f0b7dc546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.406722 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-8437a1fb-6341-44d3-ae8d-752f0b7dc546 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:37.408535 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8437a1fb-6341-44d3-ae8d-752f0b7dc546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:37.412470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3bd26-c982-49e4-835f-f9c020b97cb3 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.413007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3bd26-c982-49e4-835f-f9c020b97cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.413484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3bd26-c982-49e4-835f-f9c020b97cb3 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.415038 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 34/63] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:37.422187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196e80a-1bd2-46f3-b518-1e64ae93f45e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.422703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196e80a-1bd2-46f3-b518-1e64ae93f45e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.423169 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196e80a-1bd2-46f3-b518-1e64ae93f45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.426287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:37.451539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e196e80a-1bd2-46f3-b518-1e64ae93f45e None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:37.452819 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e196e80a-1bd2-46f3-b518-1e64ae93f45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:37.521911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 30/64] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:37.523648 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b99b8d-be33-44b8-8632-0bf240fac68d None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.532023 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b99b8d-be33-44b8-8632-0bf240fac68d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.532715 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b99b8d-be33-44b8-8632-0bf240fac68d None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.534400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 31/65] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:37 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:37.545296 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ed898-2c1c-420e-9929-7ad00f938e3b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.545584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ed898-2c1c-420e-9929-7ad00f938e3b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.545867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ed898-2c1c-420e-9929-7ad00f938e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.555277 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 35/66] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:37.561732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196e80a-1bd2-46f3-b518-1e64ae93f45e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:37.572036 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-c25ed898-2c1c-420e-9929-7ad00f938e3b None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:37.573420 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c25ed898-2c1c-420e-9929-7ad00f938e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:37.659707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 32/67] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:37.659707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ed898-2c1c-420e-9929-7ad00f938e3b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:37.680904 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:37.692596 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196e80a-1bd2-46f3-b518-1e64ae93f45e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:37.731724 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60564547-cb1f-4247-9b07-21b3427a3686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:37.732562 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60564547-cb1f-4247-9b07-21b3427a3686 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.732848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60564547-cb1f-4247-9b07-21b3427a3686 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.733131 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60564547-cb1f-4247-9b07-21b3427a3686 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.738196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:37.746402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60564547-cb1f-4247-9b07-21b3427a3686 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:37.754598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-60564547-cb1f-4247-9b07-21b3427a3686 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:37.754598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:37.754598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:37.754598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:37.754598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:37.754598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:37.754598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:37.754598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:37.754598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:37.754598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:37.754598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:37.754598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:37.754598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:37.754598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:37.754598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:37.754598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:37.755535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:37.755535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:37.755535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:37.755535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:37.755535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:37.755535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:37.755535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:37.755535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:37.755535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:37.755535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:37.755535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:37.755535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:37.755535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:37.755535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:37.755535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:37.755535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:37.756739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:37.756739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:37.756739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:37.756739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:37.756739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:37.756739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:37.756739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:37.756739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:37.756739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:37.756739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:37.756739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:37.756739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:02:37.756739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 36/68] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:02:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:37.757574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c638f-de40-4202-b01b-a282442f0bdf None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.757574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c638f-de40-4202-b01b-a282442f0bdf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.757698 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c638f-de40-4202-b01b-a282442f0bdf None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.759113 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 37/69] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:37.763552 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c638f-de40-4202-b01b-a282442f0bdf None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:37.768271 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ed898-2c1c-420e-9929-7ad00f938e3b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:37.811487 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73bd72-6e21-453e-843a-6a2ff00dcc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:37.812350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73bd72-6e21-453e-843a-6a2ff00dcc82 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.812642 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73bd72-6e21-453e-843a-6a2ff00dcc82 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.812915 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73bd72-6e21-453e-843a-6a2ff00dcc82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.825028 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b73bd72-6e21-453e-843a-6a2ff00dcc82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:37.833397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-8b73bd72-6e21-453e-843a-6a2ff00dcc82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:37.833397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:37.833397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:37.833397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:37.833397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:37.833397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:37.833397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:37.833397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:37.833397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:37.833397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:37.833397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:37.833397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:37.833397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:37.833397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:37.833397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:37.833397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:37.834246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:37.834246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:37.834246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:37.834246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:37.834246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:37.834246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:37.834246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:37.834246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:37.834246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:37.834246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:37.834246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:37.834246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:37.834246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:37.834246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:37.834246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:37.834246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:37.835171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:37.835171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:37.835171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:37.835171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:37.835171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:37.835171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:37.835171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:37.835171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:37.835171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:37.835171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:37.835171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:37.835171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:02:37.835171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 33/70] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:37.836011 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73bd72-6e21-453e-843a-6a2ff00dcc82 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:37.872615 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c638f-de40-4202-b01b-a282442f0bdf None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:37.905305 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee65664-992c-4d29-9796-ce2eebd25c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:37.906116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee65664-992c-4d29-9796-ce2eebd25c1c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.906399 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee65664-992c-4d29-9796-ce2eebd25c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.906671 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee65664-992c-4d29-9796-ce2eebd25c1c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.916994 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee65664-992c-4d29-9796-ce2eebd25c1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:37.925147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-aee65664-992c-4d29-9796-ce2eebd25c1c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:37.925147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:37.925147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:37.925147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:37.925147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:37.925147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:37.925147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:37.925147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:37.925147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:37.925147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:37.925147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:37.925147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:37.925147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:37.925147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:37.925147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:37.925147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:37.925985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:37.925985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:37.925985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:37.925985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:37.925985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:37.925985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:37.925985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:37.925985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:37.925985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:37.925985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:37.925985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:37.925985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:37.925985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:37.925985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:37.925985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:37.925985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:37.926919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:37.926919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:37.926919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:37.926919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:37.926919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:37.926919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:37.926919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:37.926919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:37.926919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:37.926919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:37.926919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:37.926919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:02:37.926919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 38/71] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:02:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:37.927812 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08234b8a-3e7d-4d52-9393-85f0a07dc1a2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.927812 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08234b8a-3e7d-4d52-9393-85f0a07dc1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.928024 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08234b8a-3e7d-4d52-9393-85f0a07dc1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.938631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73bd72-6e21-453e-843a-6a2ff00dcc82 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:37.944229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-08234b8a-3e7d-4d52-9393-85f0a07dc1a2 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:37.945420 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-08234b8a-3e7d-4d52-9393-85f0a07dc1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:37.952733 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:37.974457 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6283ec6-8517-451a-a408-f74eceeab206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:37.975135 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6283ec6-8517-451a-a408-f74eceeab206 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.975418 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6283ec6-8517-451a-a408-f74eceeab206 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.975729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6283ec6-8517-451a-a408-f74eceeab206 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.976487 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6283ec6-8517-451a-a408-f74eceeab206 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:37.976794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6283ec6-8517-451a-a408-f74eceeab206 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:37.978198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6283ec6-8517-451a-a408-f74eceeab206 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:37.980467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:37.992221 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 34/72] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:37.992221 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a126626-6db8-46a6-bbfb-430abcdb01e0 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.992221 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a126626-6db8-46a6-bbfb-430abcdb01e0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.992221 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a126626-6db8-46a6-bbfb-430abcdb01e0 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.993098 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 35/73] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:37.994755 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a126626-6db8-46a6-bbfb-430abcdb01e0 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:38.020628 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 39/74] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:38.021502 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08234b8a-3e7d-4d52-9393-85f0a07dc1a2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:38.114250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a126626-6db8-46a6-bbfb-430abcdb01e0 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:38.120556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08234b8a-3e7d-4d52-9393-85f0a07dc1a2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:38.148646 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedbcb8-90a0-4b2b-bbdf-52e8e702c80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:38.150337 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedbcb8-90a0-4b2b-bbdf-52e8e702c80b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:38.150625 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedbcb8-90a0-4b2b-bbdf-52e8e702c80b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:38.150906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedbcb8-90a0-4b2b-bbdf-52e8e702c80b None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:38.151700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fedbcb8-90a0-4b2b-bbdf-52e8e702c80b None admin] RBAC: Adding query filter params (name=member) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:38.152017 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fedbcb8-90a0-4b2b-bbdf-52e8e702c80b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:38.153451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fedbcb8-90a0-4b2b-bbdf-52e8e702c80b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:38.157600 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7d94d-458d-4c47-92ca-398224852496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:38.158281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7d94d-458d-4c47-92ca-398224852496 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:38.158540 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7d94d-458d-4c47-92ca-398224852496 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:38.158831 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7d94d-458d-4c47-92ca-398224852496 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:38.159583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7d94d-458d-4c47-92ca-398224852496 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:38.159923 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7d94d-458d-4c47-92ca-398224852496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:38.161356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7d94d-458d-4c47-92ca-398224852496 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:38.162372 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 36/75] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:38.163157 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedbcb8-90a0-4b2b-bbdf-52e8e702c80b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:38.174783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 40/76] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:02:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:38.176081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a9d4f-bcac-4b89-8af7-5edf60703cba None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:38.176374 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a9d4f-bcac-4b89-8af7-5edf60703cba None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:38.176652 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a9d4f-bcac-4b89-8af7-5edf60703cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:38.182550 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:38.194395 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d92a9d4f-bcac-4b89-8af7-5edf60703cba None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:38.195634 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d92a9d4f-bcac-4b89-8af7-5edf60703cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:38.228014 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:38.263244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 41/77] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:38.271762 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a9d4f-bcac-4b89-8af7-5edf60703cba None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:38.279780 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedbcb8-90a0-4b2b-bbdf-52e8e702c80b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:38.314744 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6887f20-b351-42af-9528-cefa971ae725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:38.315674 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6887f20-b351-42af-9528-cefa971ae725 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:38.315965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6887f20-b351-42af-9528-cefa971ae725 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:38.316248 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6887f20-b351-42af-9528-cefa971ae725 None admin] PATH_INFO: `/v3/users/3dc250d3d60a40d8b744430d137fcb9c` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:38.317493 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6887f20-b351-42af-9528-cefa971ae725 None admin] RBAC: Authorizing `identity:get_user(user_id=3dc250d3d60a40d8b744430d137fcb9c)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:38.318992 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6887f20-b351-42af-9528-cefa971ae725 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:38.320692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 37/78] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:38 2022] GET /identity/v3/users/3dc250d3d60a40d8b744430d137fcb9c => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:38.321452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6887f20-b351-42af-9528-cefa971ae725 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:38.368284 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a9d4f-bcac-4b89-8af7-5edf60703cba None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:38.401831 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f25d5e-380c-4b65-9449-06c684a0bc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:38.402650 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f25d5e-380c-4b65-9449-06c684a0bc6b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:38.402937 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f25d5e-380c-4b65-9449-06c684a0bc6b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:38.403204 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f25d5e-380c-4b65-9449-06c684a0bc6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:38.405236 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:38.415386 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f25d5e-380c-4b65-9449-06c684a0bc6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:38.422211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6887f20-b351-42af-9528-cefa971ae725 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:38.423229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-e1f25d5e-380c-4b65-9449-06c684a0bc6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:38.423229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:38.423229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:38.423229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:38.423229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:38.423229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:38.423229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:38.423229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:38.423229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:38.423229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:38.423229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:38.423229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:38.423229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:38.423229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:38.423229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:38.423229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:38.424230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:38.424230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:38.424230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:38.424230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:38.424230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:38.424230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:38.424230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:38.424230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:38.424230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:38.424230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:38.424230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:38.424230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:38.424230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:38.424230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:38.424230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:38.424230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:38.425143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:38.425143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:38.425143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:38.425143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:38.425143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:38.425143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:38.425143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:38.425143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:38.425143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:38.425143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:38.425143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:38.425143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:02:38.425143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 42/79] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:38.425777 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f25d5e-380c-4b65-9449-06c684a0bc6b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:38.450492 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:38.459279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529588dd-db4f-4a1d-a89f-2c5dcb44795c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:38.460215 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529588dd-db4f-4a1d-a89f-2c5dcb44795c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:38.460503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529588dd-db4f-4a1d-a89f-2c5dcb44795c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:38.460784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529588dd-db4f-4a1d-a89f-2c5dcb44795c None admin] PATH_INFO: `/v3/users/a393cbda710c4f2ca8a06ed49b1c5e5b` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:38.477577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529588dd-db4f-4a1d-a89f-2c5dcb44795c None admin] RBAC: Authorizing `identity:get_user(user_id=a393cbda710c4f2ca8a06ed49b1c5e5b)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:38.479145 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529588dd-db4f-4a1d-a89f-2c5dcb44795c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:38.481071 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 38/80] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:38 2022] GET /identity/v3/users/a393cbda710c4f2ca8a06ed49b1c5e5b => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:38.481634 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529588dd-db4f-4a1d-a89f-2c5dcb44795c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:38.528757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f25d5e-380c-4b65-9449-06c684a0bc6b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:38.565267 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bedc2b-7c65-44e3-a4ca-467b17febc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:38.566179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bedc2b-7c65-44e3-a4ca-467b17febc7c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:38.566462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bedc2b-7c65-44e3-a4ca-467b17febc7c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:38.566735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bedc2b-7c65-44e3-a4ca-467b17febc7c None admin] PATH_INFO: `/v3/users/a393cbda710c4f2ca8a06ed49b1c5e5b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:38.567959 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bedc2b-7c65-44e3-a4ca-467b17febc7c None admin] RBAC: Authorizing `identity:get_user(user_id=a393cbda710c4f2ca8a06ed49b1c5e5b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:38.569448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bedc2b-7c65-44e3-a4ca-467b17febc7c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:38.571097 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 43/81] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:38 2022] GET /identity/v3/users/a393cbda710c4f2ca8a06ed49b1c5e5b => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:38.571876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bedc2b-7c65-44e3-a4ca-467b17febc7c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:38.589521 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529588dd-db4f-4a1d-a89f-2c5dcb44795c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:38.623423 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc772e4-26eb-477d-a257-285bd8c592f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:38.624275 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc772e4-26eb-477d-a257-285bd8c592f1 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:38.624563 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc772e4-26eb-477d-a257-285bd8c592f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:38.624845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc772e4-26eb-477d-a257-285bd8c592f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:38.626048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc772e4-26eb-477d-a257-285bd8c592f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:38.627635 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc772e4-26eb-477d-a257-285bd8c592f1 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:38.629320 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 39/82] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:38.630084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc772e4-26eb-477d-a257-285bd8c592f1 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:38.637321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:38.677220 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bedc2b-7c65-44e3-a4ca-467b17febc7c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:38.687540 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:38.709572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be58e2d-12d0-479d-b3b7-15230251b513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:38.710233 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be58e2d-12d0-479d-b3b7-15230251b513 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:38.710523 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be58e2d-12d0-479d-b3b7-15230251b513 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:38.710798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be58e2d-12d0-479d-b3b7-15230251b513 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:38.735090 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc772e4-26eb-477d-a257-285bd8c592f1 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:38.769969 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0308fe6-29ba-43c6-a309-f81d99aae9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:38.770644 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0308fe6-29ba-43c6-a309-f81d99aae9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:38.770931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0308fe6-29ba-43c6-a309-f81d99aae9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:38.771212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0308fe6-29ba-43c6-a309-f81d99aae9c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:38.772023 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0308fe6-29ba-43c6-a309-f81d99aae9c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:38.772334 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0308fe6-29ba-43c6-a309-f81d99aae9c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:38.773740 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0308fe6-29ba-43c6-a309-f81d99aae9c0 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:38.777651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be58e2d-12d0-479d-b3b7-15230251b513 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:38.779130 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be58e2d-12d0-479d-b3b7-15230251b513 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:38.781743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 40/83] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:38.782848 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0308fe6-29ba-43c6-a309-f81d99aae9c0 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:38.839820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:38.879351 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 44/84] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:38.880241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be58e2d-12d0-479d-b3b7-15230251b513 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:38.887571 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0308fe6-29ba-43c6-a309-f81d99aae9c0 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:38.904242 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:38.922294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63530b11-62d1-426b-8800-ff0049920341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:38.922956 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63530b11-62d1-426b-8800-ff0049920341 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:38.923238 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63530b11-62d1-426b-8800-ff0049920341 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:38.923516 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63530b11-62d1-426b-8800-ff0049920341 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:38.978092 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be58e2d-12d0-479d-b3b7-15230251b513 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:38.990694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63530b11-62d1-426b-8800-ff0049920341 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:38.992259 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63530b11-62d1-426b-8800-ff0049920341 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:39.014215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc241e-efda-474d-8ca5-04cbcb6d1dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:39.014886 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc241e-efda-474d-8ca5-04cbcb6d1dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:39.015172 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc241e-efda-474d-8ca5-04cbcb6d1dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:39.015446 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc241e-efda-474d-8ca5-04cbcb6d1dc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:39.047270 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:39.086410 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc241e-efda-474d-8ca5-04cbcb6d1dc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:39.088414 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc241e-efda-474d-8ca5-04cbcb6d1dc3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:39.099817 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 41/85] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:39.100653 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63530b11-62d1-426b-8800-ff0049920341 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:39.118063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:39.187856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 45/86] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:39.188533 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc241e-efda-474d-8ca5-04cbcb6d1dc3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:39.207125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63530b11-62d1-426b-8800-ff0049920341 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:39.241562 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804493f-a4ee-45a8-8bff-e7463809240a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:39.242179 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804493f-a4ee-45a8-8bff-e7463809240a None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:39.242467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804493f-a4ee-45a8-8bff-e7463809240a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:39.242748 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804493f-a4ee-45a8-8bff-e7463809240a None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:39.243476 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6804493f-a4ee-45a8-8bff-e7463809240a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:39.245006 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6804493f-a4ee-45a8-8bff-e7463809240a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:39.247951 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:39.264877 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 42/87] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Jun 27 21:02:39 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:02:39.265690 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804493f-a4ee-45a8-8bff-e7463809240a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:39.289734 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc241e-efda-474d-8ca5-04cbcb6d1dc3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:39.329264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3d420-2989-492a-a3a6-5842132d54ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:39.330168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3d420-2989-492a-a3a6-5842132d54ce None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:39.330536 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3d420-2989-492a-a3a6-5842132d54ce None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:39.330811 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3d420-2989-492a-a3a6-5842132d54ce None admin] PATH_INFO: `/v3/users/a393cbda710c4f2ca8a06ed49b1c5e5b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:39.332148 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3d420-2989-492a-a3a6-5842132d54ce None admin] RBAC: Authorizing `identity:get_user(user_id=a393cbda710c4f2ca8a06ed49b1c5e5b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:39.333728 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3d420-2989-492a-a3a6-5842132d54ce None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:39.335646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 46/88] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:39 2022] GET /identity/v3/users/a393cbda710c4f2ca8a06ed49b1c5e5b => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:39.335646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3d420-2989-492a-a3a6-5842132d54ce None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:39.353614 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:39.388376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804493f-a4ee-45a8-8bff-e7463809240a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:39.426601 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32bb670-72b9-4ecb-bf34-9cee8e116cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:39.427587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bb670-72b9-4ecb-bf34-9cee8e116cda None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:39.428409 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bb670-72b9-4ecb-bf34-9cee8e116cda None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:39.428409 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bb670-72b9-4ecb-bf34-9cee8e116cda None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:39.429382 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32bb670-72b9-4ecb-bf34-9cee8e116cda None admin] RBAC: Authorizing `identity:get_project(project_id=0dc9df6fada54f72a35899fa21f03b20)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:39.431595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32bb670-72b9-4ecb-bf34-9cee8e116cda None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:39.432582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 43/89] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:39 2022] GET /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:39.433352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32bb670-72b9-4ecb-bf34-9cee8e116cda None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:39.455776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3d420-2989-492a-a3a6-5842132d54ce None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:39.486789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:39.493330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36a44d-fafe-4dcc-8515-5162b366095c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:39.494403 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36a44d-fafe-4dcc-8515-5162b366095c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:39.494813 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36a44d-fafe-4dcc-8515-5162b366095c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:39.495640 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36a44d-fafe-4dcc-8515-5162b366095c None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:39.497344 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36a44d-fafe-4dcc-8515-5162b366095c None admin] RBAC: Authorizing `identity:get_project(project_id=0dc9df6fada54f72a35899fa21f03b20)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:39.499958 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36a44d-fafe-4dcc-8515-5162b366095c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:39.500835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 47/90] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:39 2022] GET /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:39.501391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36a44d-fafe-4dcc-8515-5162b366095c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:39.560481 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32bb670-72b9-4ecb-bf34-9cee8e116cda None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:39.599099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a79f05-a410-4f62-a351-248881c0f3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:39.600042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a79f05-a410-4f62-a351-248881c0f3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:39.600857 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a79f05-a410-4f62-a351-248881c0f3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:39.600857 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a79f05-a410-4f62-a351-248881c0f3e9 None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:39.602079 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a79f05-a410-4f62-a351-248881c0f3e9 None admin] RBAC: Authorizing `identity:get_project(project_id=0dc9df6fada54f72a35899fa21f03b20)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:39.603857 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a79f05-a410-4f62-a351-248881c0f3e9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:39.605456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 44/91] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:39 2022] GET /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:39.605939 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a79f05-a410-4f62-a351-248881c0f3e9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:39.619122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:39.623035 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36a44d-fafe-4dcc-8515-5162b366095c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:39.659920 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521ec95-c60c-4b86-91d8-797c2c5df54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:39.660583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521ec95-c60c-4b86-91d8-797c2c5df54f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:39.660865 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521ec95-c60c-4b86-91d8-797c2c5df54f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:39.661141 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521ec95-c60c-4b86-91d8-797c2c5df54f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:39.714200 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a79f05-a410-4f62-a351-248881c0f3e9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:39.728936 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:39.732507 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521ec95-c60c-4b86-91d8-797c2c5df54f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:39.734072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521ec95-c60c-4b86-91d8-797c2c5df54f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:39.757314 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e99ab0-65e2-44f7-9843-91b8138d3241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:39.757913 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e99ab0-65e2-44f7-9843-91b8138d3241 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:39.758208 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e99ab0-65e2-44f7-9843-91b8138d3241 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:39.758493 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e99ab0-65e2-44f7-9843-91b8138d3241 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:39.759533 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e99ab0-65e2-44f7-9843-91b8138d3241 None admin] RBAC: Adding query filter params (role.id=5c3c095bb53943eca95c4e049cd8cbf5, scope.project.id=0dc9df6fada54f72a35899fa21f03b20, user.id=3dc250d3d60a40d8b744430d137fcb9c) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:39.759892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e99ab0-65e2-44f7-9843-91b8138d3241 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:39.761362 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e99ab0-65e2-44f7-9843-91b8138d3241 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:39.770321 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 45/92] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:39 2022] GET /identity/v3/role_assignments?user.id=3dc250d3d60a40d8b744430d137fcb9c&scope.project.id=0dc9df6fada54f72a35899fa21f03b20&role.id=5c3c095bb53943eca95c4e049cd8cbf5 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:39.770977 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e99ab0-65e2-44f7-9843-91b8138d3241 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:39.850426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:39.896418 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e99ab0-65e2-44f7-9843-91b8138d3241 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:39.907051 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 48/93] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:39.908303 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521ec95-c60c-4b86-91d8-797c2c5df54f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:39.964508 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e081ec-ba0f-4d21-8a76-9591ebb6c618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:39.965213 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e081ec-ba0f-4d21-8a76-9591ebb6c618 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:39.965503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e081ec-ba0f-4d21-8a76-9591ebb6c618 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:39.965784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e081ec-ba0f-4d21-8a76-9591ebb6c618 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:39.966646 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e081ec-ba0f-4d21-8a76-9591ebb6c618 None admin] RBAC: Adding query filter params (role.id=b1bcb7445ee54636ae7b29a2485cc201, scope.project.id=0dc9df6fada54f72a35899fa21f03b20, user.id=a393cbda710c4f2ca8a06ed49b1c5e5b) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:39.966964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e081ec-ba0f-4d21-8a76-9591ebb6c618 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:39.968500 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e081ec-ba0f-4d21-8a76-9591ebb6c618 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:39.983974 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 46/94] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:39 2022] GET /identity/v3/role_assignments?user.id=a393cbda710c4f2ca8a06ed49b1c5e5b&scope.project.id=0dc9df6fada54f72a35899fa21f03b20&role.id=b1bcb7445ee54636ae7b29a2485cc201 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:39.984855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e081ec-ba0f-4d21-8a76-9591ebb6c618 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:40.108737 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:40.119765 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521ec95-c60c-4b86-91d8-797c2c5df54f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:40.159637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e081ec-ba0f-4d21-8a76-9591ebb6c618 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:40.186105 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02860b00-8e46-4e83-82b7-4e5b93614cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:40.186796 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02860b00-8e46-4e83-82b7-4e5b93614cee None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:40.187080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02860b00-8e46-4e83-82b7-4e5b93614cee None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:40.187356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02860b00-8e46-4e83-82b7-4e5b93614cee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:40.188232 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02860b00-8e46-4e83-82b7-4e5b93614cee None admin] RBAC: Adding query filter params (role.id=b3efb2b2e43c4af8b1cf36a37f1e05f9, scope.project.id=0dc9df6fada54f72a35899fa21f03b20, user.id=a393cbda710c4f2ca8a06ed49b1c5e5b) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:40.192086 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02860b00-8e46-4e83-82b7-4e5b93614cee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:40.193641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02860b00-8e46-4e83-82b7-4e5b93614cee None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:40.208048 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 49/95] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:39 2022] GET /identity/v3/role_assignments?user.id=a393cbda710c4f2ca8a06ed49b1c5e5b&scope.project.id=0dc9df6fada54f72a35899fa21f03b20&role.id=b3efb2b2e43c4af8b1cf36a37f1e05f9 => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:40.221999 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea967f2-93d0-4d57-a56f-22c25e8d632a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:40.222919 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea967f2-93d0-4d57-a56f-22c25e8d632a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:40.223204 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea967f2-93d0-4d57-a56f-22c25e8d632a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:40.223483 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea967f2-93d0-4d57-a56f-22c25e8d632a None admin] PATH_INFO: `/v3/projects/0e99c982fb1c4cce8a268f9ce2c3383f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:40.224986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea967f2-93d0-4d57-a56f-22c25e8d632a None admin] RBAC: Authorizing `identity:get_project(project_id=0e99c982fb1c4cce8a268f9ce2c3383f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:40.226556 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea967f2-93d0-4d57-a56f-22c25e8d632a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:40.228572 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 47/96] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:39 2022] GET /identity/v3/projects/0e99c982fb1c4cce8a268f9ce2c3383f => generated 312 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:40.234578 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02860b00-8e46-4e83-82b7-4e5b93614cee None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:40.349136 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02860b00-8e46-4e83-82b7-4e5b93614cee None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:40.401931 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fed3fd-4fe5-48ce-97ad-de0528916433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:40.402569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fed3fd-4fe5-48ce-97ad-de0528916433 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:40.402855 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fed3fd-4fe5-48ce-97ad-de0528916433 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:40.403129 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fed3fd-4fe5-48ce-97ad-de0528916433 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:40.404007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fed3fd-4fe5-48ce-97ad-de0528916433 None admin] RBAC: Adding query filter params (role.id=b1bcb7445ee54636ae7b29a2485cc201, scope.project.id=0e99c982fb1c4cce8a268f9ce2c3383f, user.id=a393cbda710c4f2ca8a06ed49b1c5e5b) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:40.404321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fed3fd-4fe5-48ce-97ad-de0528916433 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:40.405791 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fed3fd-4fe5-48ce-97ad-de0528916433 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:40.416568 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 50/97] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:40 2022] GET /identity/v3/role_assignments?user.id=a393cbda710c4f2ca8a06ed49b1c5e5b&scope.project.id=0e99c982fb1c4cce8a268f9ce2c3383f&role.id=b1bcb7445ee54636ae7b29a2485cc201 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:42.004940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea756b-62d6-4d50-96e5-2013f0e01e6a None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:42.005981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea756b-62d6-4d50-96e5-2013f0e01e6a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:42.006460 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea756b-62d6-4d50-96e5-2013f0e01e6a None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:42.008322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 48/98] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:42.014872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5490a98-7a39-4f74-bc35-d9ebd7fd001c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:42.015794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5490a98-7a39-4f74-bc35-d9ebd7fd001c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:42.016282 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5490a98-7a39-4f74-bc35-d9ebd7fd001c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:42.043560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d5490a98-7a39-4f74-bc35-d9ebd7fd001c None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:42.045057 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d5490a98-7a39-4f74-bc35-d9ebd7fd001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:42.073635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:42.133917 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 51/99] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:42.169189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea756b-62d6-4d50-96e5-2013f0e01e6a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:42.173612 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:42.277904 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea756b-62d6-4d50-96e5-2013f0e01e6a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:42.289436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fd3fb-e2dd-4707-9dda-93427c83d6e7 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:42.289723 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fd3fb-e2dd-4707-9dda-93427c83d6e7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:42.290000 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fd3fb-e2dd-4707-9dda-93427c83d6e7 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:42.291390 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 52/100] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:42.310472 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af0602-2b81-45d8-8e23-2de2ed4e7c94 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:42.310804 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af0602-2b81-45d8-8e23-2de2ed4e7c94 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:42.310992 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af0602-2b81-45d8-8e23-2de2ed4e7c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:42.328315 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19735374-b5b1-41f4-98f0-3ce831226cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:42.329159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19735374-b5b1-41f4-98f0-3ce831226cab None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:42.329436 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19735374-b5b1-41f4-98f0-3ce831226cab None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:42.329713 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19735374-b5b1-41f4-98f0-3ce831226cab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:42.330757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a4af0602-2b81-45d8-8e23-2de2ed4e7c94 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:42.330927 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19735374-b5b1-41f4-98f0-3ce831226cab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:42.332319 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a4af0602-2b81-45d8-8e23-2de2ed4e7c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:42.332420 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19735374-b5b1-41f4-98f0-3ce831226cab None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:42.334059 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 49/101] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:42.339448 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19735374-b5b1-41f4-98f0-3ce831226cab None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:42.444219 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 53/102] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:42.461827 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d0fd4-4fbc-4cd1-b45e-6fc48d10288c None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:42.462433 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d0fd4-4fbc-4cd1-b45e-6fc48d10288c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:42.462898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d0fd4-4fbc-4cd1-b45e-6fc48d10288c None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:42.464519 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 54/103] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:42.467728 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:42.470626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43e11b-ef3f-4fe8-be04-83334a11101f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:42.471212 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43e11b-ef3f-4fe8-be04-83334a11101f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:42.471699 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43e11b-ef3f-4fe8-be04-83334a11101f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:42.499307 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-da43e11b-ef3f-4fe8-be04-83334a11101f None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:42.500863 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-da43e11b-ef3f-4fe8-be04-83334a11101f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:42.540437 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19735374-b5b1-41f4-98f0-3ce831226cab None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:42.603020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380feafd-0740-4f06-9981-b9b30cdf33e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:42.604163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380feafd-0740-4f06-9981-b9b30cdf33e9 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:42.604640 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380feafd-0740-4f06-9981-b9b30cdf33e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:42.605104 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380feafd-0740-4f06-9981-b9b30cdf33e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:42.606006 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380feafd-0740-4f06-9981-b9b30cdf33e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:42.608420 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380feafd-0740-4f06-9981-b9b30cdf33e9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:42.628981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 55/104] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:42.630294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43e11b-ef3f-4fe8-be04-83334a11101f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:42.655870 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 50/105] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Jun 27 21:02:42 2022] POST /identity/v3/projects => generated 302 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:02:42.670141 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:42.671862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380feafd-0740-4f06-9981-b9b30cdf33e9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:42.890350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380feafd-0740-4f06-9981-b9b30cdf33e9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:42.891768 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43e11b-ef3f-4fe8-be04-83334a11101f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:42.916504 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:42.931872 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ec043-e62f-473c-8fd0-1046498caa11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:42.932924 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ec043-e62f-473c-8fd0-1046498caa11 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:42.933379 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ec043-e62f-473c-8fd0-1046498caa11 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:42.933801 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ec043-e62f-473c-8fd0-1046498caa11 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:42.945425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ec043-e62f-473c-8fd0-1046498caa11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:42.954293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-3e0ec043-e62f-473c-8fd0-1046498caa11 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:42.954293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:42.954293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:42.954293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:42.954293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:42.954293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:42.954293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:42.954293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:42.954293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:42.954293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:42.954293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:42.954293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:42.954293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:42.954293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:42.954293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:42.954293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:42.955414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:42.955414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:42.955414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:42.955414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:42.955414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:42.955414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:42.955414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:42.955414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:42.955414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:42.955414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:42.955414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:42.955414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:42.955414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:42.955414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:42.955414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:42.955414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:42.956285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:42.956285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:42.956285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:42.956285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:42.956285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:42.956285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:42.956285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:42.956285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:42.956285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:42.956285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:42.956285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:42.956285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:02:42.957554 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 51/106] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:42.962957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ec043-e62f-473c-8fd0-1046498caa11 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:42.972948 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c67bc-ac4d-4cd1-99a8-c62f1fcc63b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:42.973793 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c67bc-ac4d-4cd1-99a8-c62f1fcc63b1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:42.974077 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c67bc-ac4d-4cd1-99a8-c62f1fcc63b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:42.974352 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c67bc-ac4d-4cd1-99a8-c62f1fcc63b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:42.997921 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c67bc-ac4d-4cd1-99a8-c62f1fcc63b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:43.010470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-a25c67bc-ac4d-4cd1-99a8-c62f1fcc63b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:43.010470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:43.010470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:43.010470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:43.010470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:43.010470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:43.010470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:43.010470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:43.010470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:43.010470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:43.010470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:43.010470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:43.010470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:43.010470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:43.010470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:43.010470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:43.013002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:43.013002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:43.013002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:43.013002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:43.013002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:43.013002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:43.013002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:43.013002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:43.013002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:43.013002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:43.013002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:43.013002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:43.013002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:43.013002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:43.013002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:43.013002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:43.017941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:43.017941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:43.017941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:43.017941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:43.017941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:43.017941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:43.017941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:43.017941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:43.017941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:43.017941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:43.017941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:43.017941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:02:43.017941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 56/107] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:02:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:43.020679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c67bc-ac4d-4cd1-99a8-c62f1fcc63b1 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:43.187727 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:43.193566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ec043-e62f-473c-8fd0-1046498caa11 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:43.234452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c67bc-ac4d-4cd1-99a8-c62f1fcc63b1 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:43.244313 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c68997-a326-40b2-a43b-a91289e1da1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:43.245008 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c68997-a326-40b2-a43b-a91289e1da1d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.245293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c68997-a326-40b2-a43b-a91289e1da1d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.245572 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c68997-a326-40b2-a43b-a91289e1da1d None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.246336 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c68997-a326-40b2-a43b-a91289e1da1d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:43.246649 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c68997-a326-40b2-a43b-a91289e1da1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:43.248075 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c68997-a326-40b2-a43b-a91289e1da1d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:43.270138 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb09f07-eee1-4228-ae7b-c52d1e039b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:43.270828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb09f07-eee1-4228-ae7b-c52d1e039b3b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.271113 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb09f07-eee1-4228-ae7b-c52d1e039b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.271391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb09f07-eee1-4228-ae7b-c52d1e039b3b None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.272190 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb09f07-eee1-4228-ae7b-c52d1e039b3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:43.272500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb09f07-eee1-4228-ae7b-c52d1e039b3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:43.273793 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 52/108] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:43.273979 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb09f07-eee1-4228-ae7b-c52d1e039b3b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:43.275037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc97f5-44f2-47bb-be52-7d10e3817e86 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.275331 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc97f5-44f2-47bb-be52-7d10e3817e86 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.275636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc97f5-44f2-47bb-be52-7d10e3817e86 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.276981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 53/109] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:43.283466 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 57/110] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:43.284412 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb09f07-eee1-4228-ae7b-c52d1e039b3b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:43.284882 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793eda5-d066-4bcb-966a-5128090fe5ff None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.285178 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793eda5-d066-4bcb-966a-5128090fe5ff None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.285458 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793eda5-d066-4bcb-966a-5128090fe5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.323629 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5793eda5-d066-4bcb-966a-5128090fe5ff None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:43.324841 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5793eda5-d066-4bcb-966a-5128090fe5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:43.385507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:43.395903 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb09f07-eee1-4228-ae7b-c52d1e039b3b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:43.412400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 54/111] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:43.413142 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793eda5-d066-4bcb-966a-5128090fe5ff None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:43.441548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad940803-c583-4156-a722-ba54e1f5e87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:43.442603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad940803-c583-4156-a722-ba54e1f5e87a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.443288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad940803-c583-4156-a722-ba54e1f5e87a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.443699 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad940803-c583-4156-a722-ba54e1f5e87a None admin] PATH_INFO: `/v3/users/c188f05705be4e49b79393cc268ace22` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.462153 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:43.468217 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad940803-c583-4156-a722-ba54e1f5e87a None admin] RBAC: Authorizing `identity:get_user(user_id=c188f05705be4e49b79393cc268ace22)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:43.470900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad940803-c583-4156-a722-ba54e1f5e87a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:43.474671 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 58/112] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:43 2022] GET /identity/v3/users/c188f05705be4e49b79393cc268ace22 => generated 289 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:43.476158 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00db8aa-b2fe-450f-9d0f-fb8a24484bb1 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.476540 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00db8aa-b2fe-450f-9d0f-fb8a24484bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.476914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00db8aa-b2fe-450f-9d0f-fb8a24484bb1 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.478283 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 59/113] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:43.479084 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00db8aa-b2fe-450f-9d0f-fb8a24484bb1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:43.527516 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793eda5-d066-4bcb-966a-5128090fe5ff None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:43.564125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7c0bd-c1a7-4bd1-8fbf-150cd66f2a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:43.565043 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7c0bd-c1a7-4bd1-8fbf-150cd66f2a53 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.565322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7c0bd-c1a7-4bd1-8fbf-150cd66f2a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.565651 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7c0bd-c1a7-4bd1-8fbf-150cd66f2a53 None admin] PATH_INFO: `/v3/users/3dc250d3d60a40d8b744430d137fcb9c` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.566901 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7c0bd-c1a7-4bd1-8fbf-150cd66f2a53 None admin] RBAC: Authorizing `identity:get_user(user_id=3dc250d3d60a40d8b744430d137fcb9c)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:43.568429 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7c0bd-c1a7-4bd1-8fbf-150cd66f2a53 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:43.570237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 55/114] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:43 2022] GET /identity/v3/users/3dc250d3d60a40d8b744430d137fcb9c => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:43.571003 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7c0bd-c1a7-4bd1-8fbf-150cd66f2a53 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:43.607771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00db8aa-b2fe-450f-9d0f-fb8a24484bb1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:43.619776 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:43.646751 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da6070-064b-4c77-bfe1-322e7b4b8dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:43.647577 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da6070-064b-4c77-bfe1-322e7b4b8dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.647913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da6070-064b-4c77-bfe1-322e7b4b8dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.648195 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da6070-064b-4c77-bfe1-322e7b4b8dc8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.661765 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da6070-064b-4c77-bfe1-322e7b4b8dc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:43.671914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f9da6070-064b-4c77-bfe1-322e7b4b8dc8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:43.671914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:43.671914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:43.671914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:43.671914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:43.671914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:43.671914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:43.671914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:43.671914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:43.671914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:43.671914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:43.671914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:43.671914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:43.671914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:43.671914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:43.671914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:43.672732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:43.672732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:43.672732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:43.672732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:43.672732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:43.672732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:43.672732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:43.672732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:43.672732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:43.672732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:43.672732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:43.672732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:43.672732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:43.672732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:43.672732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:43.672732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:43.673516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:43.673516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:43.673516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:43.673516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:43.673516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:43.673516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:43.673516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:43.673516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:43.673516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:43.673516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:43.673516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:43.673516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:02:43.674665 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 60/115] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:43.675370 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae71414-58a1-4f5d-bb83-ff6cd4d747f8 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.678772 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7c0bd-c1a7-4bd1-8fbf-150cd66f2a53 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:43.679160 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae71414-58a1-4f5d-bb83-ff6cd4d747f8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.679160 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae71414-58a1-4f5d-bb83-ff6cd4d747f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.694695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-fae71414-58a1-4f5d-bb83-ff6cd4d747f8 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:43.695898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fae71414-58a1-4f5d-bb83-ff6cd4d747f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:43.714055 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670f308-babf-416f-887b-f3b6078fb109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:43.714724 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670f308-babf-416f-887b-f3b6078fb109 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.715013 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670f308-babf-416f-887b-f3b6078fb109 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.715296 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670f308-babf-416f-887b-f3b6078fb109 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.750261 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:43.769941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 61/116] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:43.770597 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae71414-58a1-4f5d-bb83-ff6cd4d747f8 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:43.791425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670f308-babf-416f-887b-f3b6078fb109 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:43.793520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670f308-babf-416f-887b-f3b6078fb109 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:43.845543 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:43.875087 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae71414-58a1-4f5d-bb83-ff6cd4d747f8 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:43.905104 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 56/117] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:43.910476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae978e-25fb-4b8f-ac87-7f658f1a7877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:43.911200 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a162865-ef19-4e45-8a30-14ce57165982 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.911494 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a162865-ef19-4e45-8a30-14ce57165982 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.911705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae978e-25fb-4b8f-ac87-7f658f1a7877 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.911795 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a162865-ef19-4e45-8a30-14ce57165982 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.911864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae978e-25fb-4b8f-ac87-7f658f1a7877 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.912090 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae978e-25fb-4b8f-ac87-7f658f1a7877 None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.913221 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 57/118] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:43.913318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae978e-25fb-4b8f-ac87-7f658f1a7877 None admin] RBAC: Authorizing `identity:get_project(project_id=0dc9df6fada54f72a35899fa21f03b20)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:43.914054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a162865-ef19-4e45-8a30-14ce57165982 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:43.914761 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae978e-25fb-4b8f-ac87-7f658f1a7877 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:43.916789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 62/119] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:43 2022] GET /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:43.917558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae978e-25fb-4b8f-ac87-7f658f1a7877 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:43.978482 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:44.033996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a162865-ef19-4e45-8a30-14ce57165982 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:44.035894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae978e-25fb-4b8f-ac87-7f658f1a7877 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:44.071320 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b86301-3d3d-40cc-8d66-d97eb519f2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:44.072366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b86301-3d3d-40cc-8d66-d97eb519f2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.072837 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b86301-3d3d-40cc-8d66-d97eb519f2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.073298 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b86301-3d3d-40cc-8d66-d97eb519f2c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.074259 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b86301-3d3d-40cc-8d66-d97eb519f2c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:44.074661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b86301-3d3d-40cc-8d66-d97eb519f2c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:44.076245 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b86301-3d3d-40cc-8d66-d97eb519f2c3 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:44.079877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc616fa9-3462-4b77-9773-331827a6b34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:44.080750 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc616fa9-3462-4b77-9773-331827a6b34a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.081037 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc616fa9-3462-4b77-9773-331827a6b34a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.081312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc616fa9-3462-4b77-9773-331827a6b34a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.090101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 58/120] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:44.090101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b86301-3d3d-40cc-8d66-d97eb519f2c3 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:44.095534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:44.096695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc616fa9-3462-4b77-9773-331827a6b34a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:44.104894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-dc616fa9-3462-4b77-9773-331827a6b34a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:44.104894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:44.104894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:44.104894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:44.104894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:44.104894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:44.104894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:44.104894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:44.104894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:44.104894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:44.104894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:44.104894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:44.104894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:44.104894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:44.104894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:44.104894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:44.105846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:44.105846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:44.105846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:44.105846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:44.105846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:44.105846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:44.105846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:44.105846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:44.105846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:44.105846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:44.105846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:44.105846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:44.105846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:44.105846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:44.105846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:44.105846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:44.112203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:44.112203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:44.112203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:44.112203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:44.112203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:44.112203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:44.112203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:44.112203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:44.112203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:44.112203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:44.112203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:44.112203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:02:44.112203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 63/121] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:44.112203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae04cca-10f5-468b-bd03-4ace8c87b2e6 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.112955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae04cca-10f5-468b-bd03-4ace8c87b2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.112955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae04cca-10f5-468b-bd03-4ace8c87b2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.128997 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-fae04cca-10f5-468b-bd03-4ace8c87b2e6 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:44.130217 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fae04cca-10f5-468b-bd03-4ace8c87b2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:44.196994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b86301-3d3d-40cc-8d66-d97eb519f2c3 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:44.199432 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 64/122] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:44.208619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae04cca-10f5-468b-bd03-4ace8c87b2e6 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:44.242246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943c804-8be9-442a-a031-f8fa98366319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:44.243100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943c804-8be9-442a-a031-f8fa98366319 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.243415 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943c804-8be9-442a-a031-f8fa98366319 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.243719 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943c804-8be9-442a-a031-f8fa98366319 None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.244928 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943c804-8be9-442a-a031-f8fa98366319 None admin] RBAC: Authorizing `identity:get_project(project_id=0dc9df6fada54f72a35899fa21f03b20)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:44.246393 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943c804-8be9-442a-a031-f8fa98366319 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:44.252991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 59/123] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:44 2022] GET /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:44.252991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943c804-8be9-442a-a031-f8fa98366319 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:44.259683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:44.319215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae04cca-10f5-468b-bd03-4ace8c87b2e6 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:44.331098 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:44.356423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6021b9-600c-47f9-a38d-c778acfffa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:44.357346 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6021b9-600c-47f9-a38d-c778acfffa84 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.357623 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6021b9-600c-47f9-a38d-c778acfffa84 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.357917 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6021b9-600c-47f9-a38d-c778acfffa84 None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20/users/3dc250d3d60a40d8b744430d137fcb9c/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.360006 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6021b9-600c-47f9-a38d-c778acfffa84 None admin] RBAC: Authorizing `identity:create_grant(project_id=0dc9df6fada54f72a35899fa21f03b20, user_id=3dc250d3d60a40d8b744430d137fcb9c, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:44.361504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6021b9-600c-47f9-a38d-c778acfffa84 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:44.362072 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943c804-8be9-442a-a031-f8fa98366319 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:44.387703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 65/124] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:02:44 2022] PUT /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20/users/3dc250d3d60a40d8b744430d137fcb9c/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:02:44.387703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6021b9-600c-47f9-a38d-c778acfffa84 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:44.412024 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f461b-66ac-4353-8d70-f41b55946c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:44.413225 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f461b-66ac-4353-8d70-f41b55946c7f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.413514 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f461b-66ac-4353-8d70-f41b55946c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.414050 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f461b-66ac-4353-8d70-f41b55946c7f None admin] PATH_INFO: `/v3/users/a393cbda710c4f2ca8a06ed49b1c5e5b` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.415891 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f461b-66ac-4353-8d70-f41b55946c7f None admin] RBAC: Authorizing `identity:get_user(user_id=a393cbda710c4f2ca8a06ed49b1c5e5b)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:44.417412 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f461b-66ac-4353-8d70-f41b55946c7f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:44.420362 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 60/125] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:44 2022] GET /identity/v3/users/a393cbda710c4f2ca8a06ed49b1c5e5b => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:44.421054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f461b-66ac-4353-8d70-f41b55946c7f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:44.530465 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6021b9-600c-47f9-a38d-c778acfffa84 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:44.534354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:44.562222 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f461b-66ac-4353-8d70-f41b55946c7f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:44.581732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2bc09-b760-40b9-8d6d-5f47c172db41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:44.582422 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2bc09-b760-40b9-8d6d-5f47c172db41 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.582708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2bc09-b760-40b9-8d6d-5f47c172db41 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.582986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2bc09-b760-40b9-8d6d-5f47c172db41 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.583797 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2bc09-b760-40b9-8d6d-5f47c172db41 None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:44.584112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2bc09-b760-40b9-8d6d-5f47c172db41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:44.585542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2bc09-b760-40b9-8d6d-5f47c172db41 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:44.594736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 66/126] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:44.595431 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2bc09-b760-40b9-8d6d-5f47c172db41 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:44.608183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3eb4b-4e12-4988-b081-6702188f4e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:44.608183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3eb4b-4e12-4988-b081-6702188f4e7e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.608183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3eb4b-4e12-4988-b081-6702188f4e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.608183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3eb4b-4e12-4988-b081-6702188f4e7e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.622037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d3eb4b-4e12-4988-b081-6702188f4e7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:44.630951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-60d3eb4b-4e12-4988-b081-6702188f4e7e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:44.630951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:44.630951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:44.630951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:44.630951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:44.630951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:44.630951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:44.630951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:44.630951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:44.630951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:44.630951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:44.630951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:44.630951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:44.630951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:44.630951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:44.630951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:44.632070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:44.632070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:44.632070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:44.632070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:44.632070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:44.632070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:44.632070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:44.632070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:44.632070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:44.632070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:44.632070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:44.632070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:44.632070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:44.632070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:44.632070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:44.632070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:44.632946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:44.632946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:44.632946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:44.632946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:44.632946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:44.632946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:44.632946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:44.632946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:44.632946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:44.632946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:44.632946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:44.632946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:02:44.632946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 61/127] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:02:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:44.638639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3eb4b-4e12-4988-b081-6702188f4e7e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:44.644423 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:44.722053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2bc09-b760-40b9-8d6d-5f47c172db41 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:44.757182 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313dffda-13fa-4c71-a1ba-ed18f4b2aaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:44.757875 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313dffda-13fa-4c71-a1ba-ed18f4b2aaec None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.758146 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313dffda-13fa-4c71-a1ba-ed18f4b2aaec None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.758427 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313dffda-13fa-4c71-a1ba-ed18f4b2aaec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.759272 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313dffda-13fa-4c71-a1ba-ed18f4b2aaec None admin] RBAC: Adding query filter params (role.id=e71d790e22744442b6ff4af3caa1e853, scope.project.id=0dc9df6fada54f72a35899fa21f03b20, user.id=c188f05705be4e49b79393cc268ace22) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:44.759627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313dffda-13fa-4c71-a1ba-ed18f4b2aaec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:44.761093 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313dffda-13fa-4c71-a1ba-ed18f4b2aaec None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:44.767171 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 67/128] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:44 2022] GET /identity/v3/role_assignments?user.id=c188f05705be4e49b79393cc268ace22&scope.project.id=0dc9df6fada54f72a35899fa21f03b20&role.id=e71d790e22744442b6ff4af3caa1e853 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:44.767842 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313dffda-13fa-4c71-a1ba-ed18f4b2aaec None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:44.774089 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3eb4b-4e12-4988-b081-6702188f4e7e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:44.803998 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:44.822932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1412b640-25bc-405b-ba74-7d20dd254052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:44.824327 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412b640-25bc-405b-ba74-7d20dd254052 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.824851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412b640-25bc-405b-ba74-7d20dd254052 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.825342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412b640-25bc-405b-ba74-7d20dd254052 None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.833052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1412b640-25bc-405b-ba74-7d20dd254052 None admin] RBAC: Authorizing `identity:get_project(project_id=0dc9df6fada54f72a35899fa21f03b20)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:44.835949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1412b640-25bc-405b-ba74-7d20dd254052 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:44.838696 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 62/129] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:44 2022] GET /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:44.842726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1412b640-25bc-405b-ba74-7d20dd254052 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:44.933798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313dffda-13fa-4c71-a1ba-ed18f4b2aaec None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:44.977018 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c19b3-e285-4662-83b1-c26d13f350c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:44.977396 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c19b3-e285-4662-83b1-c26d13f350c2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.977685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c19b3-e285-4662-83b1-c26d13f350c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.977964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c19b3-e285-4662-83b1-c26d13f350c2 None admin] PATH_INFO: `/v3/users/a393cbda710c4f2ca8a06ed49b1c5e5b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.979225 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c19b3-e285-4662-83b1-c26d13f350c2 None admin] RBAC: Authorizing `identity:get_user(user_id=a393cbda710c4f2ca8a06ed49b1c5e5b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:44.983635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c19b3-e285-4662-83b1-c26d13f350c2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:44.983635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 68/130] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:44 2022] GET /identity/v3/users/a393cbda710c4f2ca8a06ed49b1c5e5b => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:44.983963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c19b3-e285-4662-83b1-c26d13f350c2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:44.999962 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:45.044755 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1412b640-25bc-405b-ba74-7d20dd254052 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:45.089046 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a04ae-0d24-47c6-91a9-60562d958ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:45.089648 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a04ae-0d24-47c6-91a9-60562d958ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:45.089935 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a04ae-0d24-47c6-91a9-60562d958ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:45.090212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a04ae-0d24-47c6-91a9-60562d958ae6 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:45.090974 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a04ae-0d24-47c6-91a9-60562d958ae6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:45.091282 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a04ae-0d24-47c6-91a9-60562d958ae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:45.092985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a04ae-0d24-47c6-91a9-60562d958ae6 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:45.108891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:45.114829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 63/131] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:02:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:45.115544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a04ae-0d24-47c6-91a9-60562d958ae6 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:45.115785 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c19b3-e285-4662-83b1-c26d13f350c2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:45.156916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60834bea-e833-4f3c-9b1b-dafbfee3fe4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:45.158974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60834bea-e833-4f3c-9b1b-dafbfee3fe4c None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:45.159471 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60834bea-e833-4f3c-9b1b-dafbfee3fe4c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:45.159969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60834bea-e833-4f3c-9b1b-dafbfee3fe4c None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20/users/a393cbda710c4f2ca8a06ed49b1c5e5b/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:45.163203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60834bea-e833-4f3c-9b1b-dafbfee3fe4c None admin] RBAC: Authorizing `identity:create_grant(project_id=0dc9df6fada54f72a35899fa21f03b20, user_id=a393cbda710c4f2ca8a06ed49b1c5e5b, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:45.165111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60834bea-e833-4f3c-9b1b-dafbfee3fe4c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:45.191726 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 69/132] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:02:44 2022] PUT /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20/users/a393cbda710c4f2ca8a06ed49b1c5e5b/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:02:45.194231 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daea13d-eed7-41c1-826d-38a1583288aa None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:45.194842 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daea13d-eed7-41c1-826d-38a1583288aa None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:45.195333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daea13d-eed7-41c1-826d-38a1583288aa None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:45.197677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 70/133] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:45.203173 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daea13d-eed7-41c1-826d-38a1583288aa None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:45.269950 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a04ae-0d24-47c6-91a9-60562d958ae6 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:45.324009 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:45.334191 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daea13d-eed7-41c1-826d-38a1583288aa None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:45.336704 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a08281-fa18-49fd-843e-8a1075a4ab93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:45.337615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a08281-fa18-49fd-843e-8a1075a4ab93 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:45.337896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a08281-fa18-49fd-843e-8a1075a4ab93 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:45.338172 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a08281-fa18-49fd-843e-8a1075a4ab93 None admin] PATH_INFO: `/v3/projects/0e99c982fb1c4cce8a268f9ce2c3383f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:45.344691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a08281-fa18-49fd-843e-8a1075a4ab93 None admin] RBAC: Authorizing `identity:get_project(project_id=0e99c982fb1c4cce8a268f9ce2c3383f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:45.346201 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a08281-fa18-49fd-843e-8a1075a4ab93 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:45.348089 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 64/134] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:45 2022] GET /identity/v3/projects/0e99c982fb1c4cce8a268f9ce2c3383f => generated 312 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:45.349345 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc3920-d141-4fc1-bb3d-946442ce226c None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:45.349637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc3920-d141-4fc1-bb3d-946442ce226c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:45.349918 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc3920-d141-4fc1-bb3d-946442ce226c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:45.381714 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d0ff2-228d-4873-9c87-4d24fa8f4a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:45.382639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d0ff2-228d-4873-9c87-4d24fa8f4a53 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:45.382918 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d0ff2-228d-4873-9c87-4d24fa8f4a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:45.383192 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d0ff2-228d-4873-9c87-4d24fa8f4a53 None admin] PATH_INFO: `/v3/users/a393cbda710c4f2ca8a06ed49b1c5e5b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:45.384563 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d0ff2-228d-4873-9c87-4d24fa8f4a53 None admin] RBAC: Authorizing `identity:get_user(user_id=a393cbda710c4f2ca8a06ed49b1c5e5b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:45.386056 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d0ff2-228d-4873-9c87-4d24fa8f4a53 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:45.387985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 71/135] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:45 2022] GET /identity/v3/users/a393cbda710c4f2ca8a06ed49b1c5e5b => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:45.388749 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d0ff2-228d-4873-9c87-4d24fa8f4a53 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:45.399114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6efc3920-d141-4fc1-bb3d-946442ce226c None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:45.400374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6efc3920-d141-4fc1-bb3d-946442ce226c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:45.422883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:45.509999 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 65/136] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:45.516070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc3920-d141-4fc1-bb3d-946442ce226c None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:45.522828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d0ff2-228d-4873-9c87-4d24fa8f4a53 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:45.562834 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0ef0c-1818-4c4b-afe6-e3908ee564a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:45.564116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0ef0c-1818-4c4b-afe6-e3908ee564a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:45.564623 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0ef0c-1818-4c4b-afe6-e3908ee564a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:45.565124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0ef0c-1818-4c4b-afe6-e3908ee564a1 None admin] PATH_INFO: `/v3/projects/0e99c982fb1c4cce8a268f9ce2c3383f/users/a393cbda710c4f2ca8a06ed49b1c5e5b/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:45.567392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0ef0c-1818-4c4b-afe6-e3908ee564a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e99c982fb1c4cce8a268f9ce2c3383f, user_id=a393cbda710c4f2ca8a06ed49b1c5e5b, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:45.569786 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0ef0c-1818-4c4b-afe6-e3908ee564a1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:45.599535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 72/137] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:02:45 2022] PUT /identity/v3/projects/0e99c982fb1c4cce8a268f9ce2c3383f/users/a393cbda710c4f2ca8a06ed49b1c5e5b/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:02:45.600673 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0ef0c-1818-4c4b-afe6-e3908ee564a1 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:45.721453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc3920-d141-4fc1-bb3d-946442ce226c None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:45.780854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:45.796029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:45.797987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78d425-5b44-499f-a928-b69ac46f0d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:45.798966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78d425-5b44-499f-a928-b69ac46f0d23 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:45.799397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78d425-5b44-499f-a928-b69ac46f0d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:45.799706 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78d425-5b44-499f-a928-b69ac46f0d23 None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:45.802098 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d78d425-5b44-499f-a928-b69ac46f0d23 None admin] RBAC: Authorizing `identity:get_project(project_id=0dc9df6fada54f72a35899fa21f03b20)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:45.804255 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d78d425-5b44-499f-a928-b69ac46f0d23 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:45.808525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 66/138] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:45 2022] GET /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20 => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:45.818064 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78d425-5b44-499f-a928-b69ac46f0d23 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:45.820400 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0ef0c-1818-4c4b-afe6-e3908ee564a1 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:45.894226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7c338-c9e3-48ed-b496-912796118ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:45.895046 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7c338-c9e3-48ed-b496-912796118ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:45.895338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7c338-c9e3-48ed-b496-912796118ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:45.895695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7c338-c9e3-48ed-b496-912796118ca9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:45.898157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7c338-c9e3-48ed-b496-912796118ca9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:45.899679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7c338-c9e3-48ed-b496-912796118ca9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:45.901578 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 73/139] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:45.906864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7c338-c9e3-48ed-b496-912796118ca9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:46.009638 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78d425-5b44-499f-a928-b69ac46f0d23 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:46.057695 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7176a04-8e71-4d12-8763-f3104d2b394f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:46.059014 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7176a04-8e71-4d12-8763-f3104d2b394f None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:46.059303 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7176a04-8e71-4d12-8763-f3104d2b394f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:46.059582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7176a04-8e71-4d12-8763-f3104d2b394f None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20/users/a393cbda710c4f2ca8a06ed49b1c5e5b/roles/b3efb2b2e43c4af8b1cf36a37f1e05f9` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:46.066544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7176a04-8e71-4d12-8763-f3104d2b394f None admin] RBAC: Authorizing `identity:create_grant(project_id=0dc9df6fada54f72a35899fa21f03b20, user_id=a393cbda710c4f2ca8a06ed49b1c5e5b, role_id=b3efb2b2e43c4af8b1cf36a37f1e05f9)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:46.068416 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7176a04-8e71-4d12-8763-f3104d2b394f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:46.099741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 67/140] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:02:45 2022] PUT /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20/users/a393cbda710c4f2ca8a06ed49b1c5e5b/roles/b3efb2b2e43c4af8b1cf36a37f1e05f9 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:02:46.114471 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7c338-c9e3-48ed-b496-912796118ca9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:46.169142 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ead0a3-88ad-4c6b-9cee-cc11d417ecca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:46.169744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ead0a3-88ad-4c6b-9cee-cc11d417ecca None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:46.170032 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ead0a3-88ad-4c6b-9cee-cc11d417ecca None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:46.170337 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ead0a3-88ad-4c6b-9cee-cc11d417ecca None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:46.171127 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ead0a3-88ad-4c6b-9cee-cc11d417ecca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:46.172728 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ead0a3-88ad-4c6b-9cee-cc11d417ecca None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:46.209762 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 74/141] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Jun 27 21:02:45 2022] POST /identity/v3/users => generated 283 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:02:46.816531 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f5dba-451f-4907-aec4-131ddb3b778e None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:46.817055 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f5dba-451f-4907-aec4-131ddb3b778e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:46.817126 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f5dba-451f-4907-aec4-131ddb3b778e None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:46.818735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 68/142] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:46.824676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8d885-cf7b-4540-a801-5790d0dfc29f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:46.824960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8d885-cf7b-4540-a801-5790d0dfc29f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:46.825242 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8d885-cf7b-4540-a801-5790d0dfc29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:46.852673 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b4a8d885-cf7b-4540-a801-5790d0dfc29f None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:46.853900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b4a8d885-cf7b-4540-a801-5790d0dfc29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:46.857038 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:46.932230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 75/143] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:46.966861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f5dba-451f-4907-aec4-131ddb3b778e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:47.036194 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:47.083870 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f5dba-451f-4907-aec4-131ddb3b778e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:47.126399 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27d1e8-e10a-486c-9402-f7c106c57d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:47.127217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27d1e8-e10a-486c-9402-f7c106c57d53 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:47.127521 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27d1e8-e10a-486c-9402-f7c106c57d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:47.127894 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27d1e8-e10a-486c-9402-f7c106c57d53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:47.138765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27d1e8-e10a-486c-9402-f7c106c57d53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:47.146982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-ad27d1e8-e10a-486c-9402-f7c106c57d53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:47.146982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:47.146982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:47.146982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:47.146982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:47.146982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:47.146982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:47.146982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:47.146982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:47.146982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:47.146982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:47.146982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:47.146982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:47.146982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:47.146982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:47.146982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:47.147873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:47.147873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:47.147873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:47.147873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:47.147873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:47.147873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:47.147873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:47.147873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:47.147873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:47.147873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:47.147873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:47.147873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:47.147873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:47.147873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:47.147873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:47.147873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:47.148623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:47.148623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:47.148623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:47.148623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:47.148623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:47.148623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:47.148623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:47.148623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:47.148623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:47.148623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:47.148623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:47.148623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:02:47.148623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 69/144] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:02:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:47.154185 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8d885-cf7b-4540-a801-5790d0dfc29f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:47.261357 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8d885-cf7b-4540-a801-5790d0dfc29f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:47.298981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363195b6-06ec-4887-ba22-5f877dc9d1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:47.299737 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363195b6-06ec-4887-ba22-5f877dc9d1c5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:47.300020 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363195b6-06ec-4887-ba22-5f877dc9d1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:47.300295 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363195b6-06ec-4887-ba22-5f877dc9d1c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:47.301048 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363195b6-06ec-4887-ba22-5f877dc9d1c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:47.301349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363195b6-06ec-4887-ba22-5f877dc9d1c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:47.302750 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363195b6-06ec-4887-ba22-5f877dc9d1c5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:47.311314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 76/145] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:47.317093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27d1e8-e10a-486c-9402-f7c106c57d53 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:47.439468 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27d1e8-e10a-486c-9402-f7c106c57d53 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:47.462356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:47.481571 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3118963-c7e8-48d6-99d2-9dc38e0b2ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:47.482476 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3118963-c7e8-48d6-99d2-9dc38e0b2ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:47.482757 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3118963-c7e8-48d6-99d2-9dc38e0b2ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:47.483036 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3118963-c7e8-48d6-99d2-9dc38e0b2ba7 None admin] PATH_INFO: `/v3/users/3dc250d3d60a40d8b744430d137fcb9c` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:47.484296 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3118963-c7e8-48d6-99d2-9dc38e0b2ba7 None admin] RBAC: Authorizing `identity:get_user(user_id=3dc250d3d60a40d8b744430d137fcb9c)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:47.485782 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3118963-c7e8-48d6-99d2-9dc38e0b2ba7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:47.491794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 70/146] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:47 2022] GET /identity/v3/users/3dc250d3d60a40d8b744430d137fcb9c => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:47.493453 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363195b6-06ec-4887-ba22-5f877dc9d1c5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:47.496432 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:47.616861 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363195b6-06ec-4887-ba22-5f877dc9d1c5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:47.659700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914ca7f-32c3-4ba2-ae2e-c537bdc804c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:47.660364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914ca7f-32c3-4ba2-ae2e-c537bdc804c3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:47.660643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914ca7f-32c3-4ba2-ae2e-c537bdc804c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:47.660915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914ca7f-32c3-4ba2-ae2e-c537bdc804c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:47.733615 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914ca7f-32c3-4ba2-ae2e-c537bdc804c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:47.735022 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914ca7f-32c3-4ba2-ae2e-c537bdc804c3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:47.737021 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:47.843753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 77/147] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:47.849345 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3118963-c7e8-48d6-99d2-9dc38e0b2ba7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:47.864915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d408489-4261-4e5f-85ea-a73c52f9f100 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:47.865198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d408489-4261-4e5f-85ea-a73c52f9f100 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:47.865466 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d408489-4261-4e5f-85ea-a73c52f9f100 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:47.866856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 78/148] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:47.873203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c2d16-3ddc-4e97-acf7-97e045119928 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:47.873483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c2d16-3ddc-4e97-acf7-97e045119928 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:47.873758 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c2d16-3ddc-4e97-acf7-97e045119928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:47.900620 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-695c2d16-3ddc-4e97-acf7-97e045119928 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:47.901839 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-695c2d16-3ddc-4e97-acf7-97e045119928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:48.003291 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3118963-c7e8-48d6-99d2-9dc38e0b2ba7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:48.019308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 79/149] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:48.020800 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdeb23-0359-42b0-94b6-aa113445e8e8 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:48.021091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdeb23-0359-42b0-94b6-aa113445e8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:48.021361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdeb23-0359-42b0-94b6-aa113445e8e8 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:48.022889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 80/150] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:48.033473 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfba8f-2f1e-4750-aff3-08647d2af931 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:48.033930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfba8f-2f1e-4750-aff3-08647d2af931 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:48.034303 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfba8f-2f1e-4750-aff3-08647d2af931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:48.053528 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-49cfba8f-2f1e-4750-aff3-08647d2af931 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:48.055253 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-49cfba8f-2f1e-4750-aff3-08647d2af931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:48.071394 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd6412-d149-4566-8c76-92d1c734f107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:48.072310 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd6412-d149-4566-8c76-92d1c734f107 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:48.072600 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd6412-d149-4566-8c76-92d1c734f107 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:48.072869 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd6412-d149-4566-8c76-92d1c734f107 None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:48.075466 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd6412-d149-4566-8c76-92d1c734f107 None admin] RBAC: Authorizing `identity:get_project(project_id=0dc9df6fada54f72a35899fa21f03b20)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:48.076951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd6412-d149-4566-8c76-92d1c734f107 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:48.078816 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 71/151] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:47 2022] GET /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:48.087390 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd6412-d149-4566-8c76-92d1c734f107 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:48.100841 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:48.157791 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 81/152] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:48.158610 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfba8f-2f1e-4750-aff3-08647d2af931 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:48.212018 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:48.238254 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd6412-d149-4566-8c76-92d1c734f107 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:48.286706 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581b8cf-6dff-41d2-b24f-2a1b225b222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:48.287543 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581b8cf-6dff-41d2-b24f-2a1b225b222b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:48.287853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581b8cf-6dff-41d2-b24f-2a1b225b222b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:48.288130 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581b8cf-6dff-41d2-b24f-2a1b225b222b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:48.320180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581b8cf-6dff-41d2-b24f-2a1b225b222b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:48.328492 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfba8f-2f1e-4750-aff3-08647d2af931 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:48.336513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-7581b8cf-6dff-41d2-b24f-2a1b225b222b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:48.336513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:48.336513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:48.336513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:48.336513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:48.336513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:48.336513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:48.336513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:48.336513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:48.336513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:48.336513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:48.336513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:48.336513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:48.336513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:48.336513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:48.336513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:48.337374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:48.337374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:48.337374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:48.337374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:48.337374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:48.337374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:48.337374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:48.337374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:48.337374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:48.337374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:48.337374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:48.337374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:48.337374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:48.337374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:48.337374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:48.337374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:48.339456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:48.339456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:48.339456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:48.339456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:48.339456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:48.339456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:48.339456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:48.339456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:48.339456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:48.339456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:48.339456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:48.339456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:02:48.339456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 72/153] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:48.339456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581b8cf-6dff-41d2-b24f-2a1b225b222b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:48.370830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5e118-d389-4d32-a60e-7e0391258faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:48.371508 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5e118-d389-4d32-a60e-7e0391258faf None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:48.371833 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5e118-d389-4d32-a60e-7e0391258faf None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:48.372249 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5e118-d389-4d32-a60e-7e0391258faf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:48.373168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5e118-d389-4d32-a60e-7e0391258faf None admin] RBAC: Adding query filter params (role.id=5c3c095bb53943eca95c4e049cd8cbf5, scope.project.id=0dc9df6fada54f72a35899fa21f03b20, user.id=3dc250d3d60a40d8b744430d137fcb9c) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:48.373557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5e118-d389-4d32-a60e-7e0391258faf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:48.375108 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5e118-d389-4d32-a60e-7e0391258faf None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:48.381911 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 82/154] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:48 2022] GET /identity/v3/role_assignments?user.id=3dc250d3d60a40d8b744430d137fcb9c&scope.project.id=0dc9df6fada54f72a35899fa21f03b20&role.id=5c3c095bb53943eca95c4e049cd8cbf5 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:48.382637 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5e118-d389-4d32-a60e-7e0391258faf None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:48.416163 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:48.489673 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5e118-d389-4d32-a60e-7e0391258faf None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:48.515001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581b8cf-6dff-41d2-b24f-2a1b225b222b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:48.525856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8a2de-94f8-4072-af29-b796e0f71c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:48.526641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8a2de-94f8-4072-af29-b796e0f71c89 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:48.526934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8a2de-94f8-4072-af29-b796e0f71c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:48.527217 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8a2de-94f8-4072-af29-b796e0f71c89 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:48.528314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8a2de-94f8-4072-af29-b796e0f71c89 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:48.528314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8a2de-94f8-4072-af29-b796e0f71c89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:48.529771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8a2de-94f8-4072-af29-b796e0f71c89 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:48.539423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 83/155] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:48.545253 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8a2de-94f8-4072-af29-b796e0f71c89 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:48.557431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:48.621761 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b8ad4-8b32-4093-9be4-2a54da3581e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:48.622587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b8ad4-8b32-4093-9be4-2a54da3581e8 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:48.622871 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b8ad4-8b32-4093-9be4-2a54da3581e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:48.623148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b8ad4-8b32-4093-9be4-2a54da3581e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:48.653448 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b8ad4-8b32-4093-9be4-2a54da3581e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:48.670187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8a2de-94f8-4072-af29-b796e0f71c89 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:48.672967 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:48.674111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-bc8b8ad4-8b32-4093-9be4-2a54da3581e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:48.674111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:48.674111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:48.674111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:48.674111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:48.674111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:48.674111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:48.674111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:48.674111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:48.674111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:48.674111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:48.674111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:48.674111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:48.674111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:48.674111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:48.674111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:48.675108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:48.675108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:48.675108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:48.675108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:48.675108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:48.675108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:48.675108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:48.675108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:48.675108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:48.675108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:48.675108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:48.675108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:48.675108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:48.675108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:48.675108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:48.675108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:48.676375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:48.676375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:48.676375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:48.676375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:48.676375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:48.676375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:48.676375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:48.676375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:48.676375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:48.676375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:48.676375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:48.676375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:02:48.676375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 73/156] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:48.684632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b8ad4-8b32-4093-9be4-2a54da3581e8 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:48.724430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ff200-7801-4db7-9962-f674911fd996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:48.725359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ff200-7801-4db7-9962-f674911fd996 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:48.725650 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ff200-7801-4db7-9962-f674911fd996 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:48.725932 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ff200-7801-4db7-9962-f674911fd996 None admin] PATH_INFO: `/v3/users/c188f05705be4e49b79393cc268ace22` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:48.727251 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ff200-7801-4db7-9962-f674911fd996 None admin] RBAC: Authorizing `identity:get_user(user_id=c188f05705be4e49b79393cc268ace22)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:48.728799 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ff200-7801-4db7-9962-f674911fd996 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:48.730824 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 84/157] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:48 2022] GET /identity/v3/users/c188f05705be4e49b79393cc268ace22 => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:48.736523 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ff200-7801-4db7-9962-f674911fd996 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:48.851983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b8ad4-8b32-4093-9be4-2a54da3581e8 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:48.892727 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ff200-7801-4db7-9962-f674911fd996 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:48.908384 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b329c6-b372-4257-bc14-f7628879a71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:48.909165 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b329c6-b372-4257-bc14-f7628879a71e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:48.909450 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b329c6-b372-4257-bc14-f7628879a71e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:48.909735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b329c6-b372-4257-bc14-f7628879a71e None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:48.910505 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b329c6-b372-4257-bc14-f7628879a71e None admin] RBAC: Adding query filter params (name=member) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:48.910818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b329c6-b372-4257-bc14-f7628879a71e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:48.912269 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b329c6-b372-4257-bc14-f7628879a71e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:48.921486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 74/158] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:48.926821 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b329c6-b372-4257-bc14-f7628879a71e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:48.948353 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64d7da-952c-42bc-97b9-3fb5fd0adf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:48.949257 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64d7da-952c-42bc-97b9-3fb5fd0adf5e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:48.949547 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64d7da-952c-42bc-97b9-3fb5fd0adf5e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:48.949829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64d7da-952c-42bc-97b9-3fb5fd0adf5e None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:48.951133 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec64d7da-952c-42bc-97b9-3fb5fd0adf5e None admin] RBAC: Authorizing `identity:get_project(project_id=0dc9df6fada54f72a35899fa21f03b20)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:48.952642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec64d7da-952c-42bc-97b9-3fb5fd0adf5e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:48.954380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 85/159] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:48 2022] GET /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:48.959931 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64d7da-952c-42bc-97b9-3fb5fd0adf5e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:49.012428 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:49.019786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:49.092998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b329c6-b372-4257-bc14-f7628879a71e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:49.138301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2233b22-19cf-48f2-ab73-e888d0b1f572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:49.139118 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2233b22-19cf-48f2-ab73-e888d0b1f572 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:49.139402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2233b22-19cf-48f2-ab73-e888d0b1f572 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:49.139703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2233b22-19cf-48f2-ab73-e888d0b1f572 None admin] PATH_INFO: `/v3/users/a393cbda710c4f2ca8a06ed49b1c5e5b` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:49.141556 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2233b22-19cf-48f2-ab73-e888d0b1f572 None admin] RBAC: Authorizing `identity:get_user(user_id=a393cbda710c4f2ca8a06ed49b1c5e5b)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:49.143073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2233b22-19cf-48f2-ab73-e888d0b1f572 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:49.147585 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 75/160] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:48 2022] GET /identity/v3/users/a393cbda710c4f2ca8a06ed49b1c5e5b => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:49.158257 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64d7da-952c-42bc-97b9-3fb5fd0adf5e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:49.167181 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2233b22-19cf-48f2-ab73-e888d0b1f572 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:49.246595 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db6f10-d337-42f2-aeed-be64d4692a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:49.247515 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db6f10-d337-42f2-aeed-be64d4692a7d None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:49.247824 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db6f10-d337-42f2-aeed-be64d4692a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:49.248098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db6f10-d337-42f2-aeed-be64d4692a7d None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20/users/c188f05705be4e49b79393cc268ace22/roles/e71d790e22744442b6ff4af3caa1e853` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:49.252490 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db6f10-d337-42f2-aeed-be64d4692a7d None admin] RBAC: Authorizing `identity:create_grant(project_id=0dc9df6fada54f72a35899fa21f03b20, user_id=c188f05705be4e49b79393cc268ace22, role_id=e71d790e22744442b6ff4af3caa1e853)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:49.253981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db6f10-d337-42f2-aeed-be64d4692a7d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:49.284245 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 86/161] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:02:48 2022] PUT /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20/users/c188f05705be4e49b79393cc268ace22/roles/e71d790e22744442b6ff4af3caa1e853 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:02:49.302264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6086ffd-635b-42e4-af43-9b69f51fe2cc None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:49.302877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6086ffd-635b-42e4-af43-9b69f51fe2cc None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:49.303388 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6086ffd-635b-42e4-af43-9b69f51fe2cc None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:49.305104 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 87/162] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:49.310742 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e4066-7266-4134-a42f-baa1ea61de1d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:49.311289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e4066-7266-4134-a42f-baa1ea61de1d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:49.311811 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e4066-7266-4134-a42f-baa1ea61de1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:49.346384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-073e4066-7266-4134-a42f-baa1ea61de1d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:49.348582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-073e4066-7266-4134-a42f-baa1ea61de1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:49.354162 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2233b22-19cf-48f2-ab73-e888d0b1f572 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:49.375986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:49.430339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90ed7b-f89a-46e7-a6e9-9772924278a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:49.430920 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90ed7b-f89a-46e7-a6e9-9772924278a5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:49.431199 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90ed7b-f89a-46e7-a6e9-9772924278a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:49.431469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90ed7b-f89a-46e7-a6e9-9772924278a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:49.500859 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 88/163] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:49.528102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e4066-7266-4134-a42f-baa1ea61de1d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:49.550718 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:49.583661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90ed7b-f89a-46e7-a6e9-9772924278a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:49.583661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90ed7b-f89a-46e7-a6e9-9772924278a5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:49.713506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e4066-7266-4134-a42f-baa1ea61de1d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:49.716988 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 76/164] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 21:02:49.725446 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90ed7b-f89a-46e7-a6e9-9772924278a5 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:49.729152 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:49.785696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c45e97-06c4-4163-a349-9b5c7c757d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:49.786685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c45e97-06c4-4163-a349-9b5c7c757d27 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:49.787057 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c45e97-06c4-4163-a349-9b5c7c757d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:49.787422 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c45e97-06c4-4163-a349-9b5c7c757d27 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:49.816569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c45e97-06c4-4163-a349-9b5c7c757d27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:49.824624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-14c45e97-06c4-4163-a349-9b5c7c757d27 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:49.824624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:49.824624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:49.824624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:49.824624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:49.824624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:49.824624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:49.824624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:49.824624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:49.824624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:49.824624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:49.824624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:49.824624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:49.824624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:49.824624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:49.824624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:49.825631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:49.825631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:49.825631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:49.825631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:49.825631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:49.825631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:49.825631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:49.825631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:49.825631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:49.825631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:49.825631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:49.825631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:49.825631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:49.825631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:49.825631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:49.825631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:49.826393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:49.826393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:49.826393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:49.826393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:49.826393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:49.826393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:49.826393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:49.826393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:49.826393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:49.826393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:49.826393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:49.826393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:02:49.827226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 89/165] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:49.836715 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c45e97-06c4-4163-a349-9b5c7c757d27 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:49.864017 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90ed7b-f89a-46e7-a6e9-9772924278a5 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:49.934100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47ee5a-040a-4739-be53-fc3670dc7247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:49.936635 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47ee5a-040a-4739-be53-fc3670dc7247 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:49.938369 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47ee5a-040a-4739-be53-fc3670dc7247 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:49.938827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47ee5a-040a-4739-be53-fc3670dc7247 None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:49.941317 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47ee5a-040a-4739-be53-fc3670dc7247 None admin] RBAC: Authorizing `identity:get_project(project_id=0dc9df6fada54f72a35899fa21f03b20)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:49.945961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47ee5a-040a-4739-be53-fc3670dc7247 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:49.947915 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 77/166] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:49 2022] GET /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:49.953707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47ee5a-040a-4739-be53-fc3670dc7247 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:49.990609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:50.040317 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c45e97-06c4-4163-a349-9b5c7c757d27 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:50.092466 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:50.113288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99f247-c35e-409e-96a1-9c4922c05bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:50.113980 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99f247-c35e-409e-96a1-9c4922c05bdc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:50.114261 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99f247-c35e-409e-96a1-9c4922c05bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:50.114534 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99f247-c35e-409e-96a1-9c4922c05bdc None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:50.115303 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99f247-c35e-409e-96a1-9c4922c05bdc None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:50.115628 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99f247-c35e-409e-96a1-9c4922c05bdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:50.117631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99f247-c35e-409e-96a1-9c4922c05bdc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:50.128984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47ee5a-040a-4739-be53-fc3670dc7247 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:50.139112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 90/167] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:50.144848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99f247-c35e-409e-96a1-9c4922c05bdc None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:50.179807 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc91c22-abfa-4042-846f-48bb9a0a0fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:50.180848 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc91c22-abfa-4042-846f-48bb9a0a0fba None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:50.181332 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc91c22-abfa-4042-846f-48bb9a0a0fba None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:50.181820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc91c22-abfa-4042-846f-48bb9a0a0fba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:50.182866 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc91c22-abfa-4042-846f-48bb9a0a0fba None admin] RBAC: Adding query filter params (role.id=b1bcb7445ee54636ae7b29a2485cc201, scope.project.id=0dc9df6fada54f72a35899fa21f03b20, user.id=a393cbda710c4f2ca8a06ed49b1c5e5b) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:50.183387 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc91c22-abfa-4042-846f-48bb9a0a0fba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:50.185416 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc91c22-abfa-4042-846f-48bb9a0a0fba None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:50.193095 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 78/168] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:49 2022] GET /identity/v3/role_assignments?user.id=a393cbda710c4f2ca8a06ed49b1c5e5b&scope.project.id=0dc9df6fada54f72a35899fa21f03b20&role.id=b1bcb7445ee54636ae7b29a2485cc201 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:50.314400 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99f247-c35e-409e-96a1-9c4922c05bdc None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:50.362947 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83eddb4-6bb1-40a2-8bad-9ff24cf11481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:50.363868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83eddb4-6bb1-40a2-8bad-9ff24cf11481 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:50.364145 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83eddb4-6bb1-40a2-8bad-9ff24cf11481 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:50.364422 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83eddb4-6bb1-40a2-8bad-9ff24cf11481 None admin] PATH_INFO: `/v3/users/a393cbda710c4f2ca8a06ed49b1c5e5b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:50.368481 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83eddb4-6bb1-40a2-8bad-9ff24cf11481 None admin] RBAC: Authorizing `identity:get_user(user_id=a393cbda710c4f2ca8a06ed49b1c5e5b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:50.370033 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83eddb4-6bb1-40a2-8bad-9ff24cf11481 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:50.371946 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 91/169] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:50 2022] GET /identity/v3/users/a393cbda710c4f2ca8a06ed49b1c5e5b => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:50.378016 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc91c22-abfa-4042-846f-48bb9a0a0fba None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:50.574548 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc91c22-abfa-4042-846f-48bb9a0a0fba None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:50.617877 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab46ba5-cda3-4241-9706-0361a4ddc42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:50.618908 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab46ba5-cda3-4241-9706-0361a4ddc42b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:50.619392 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab46ba5-cda3-4241-9706-0361a4ddc42b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:50.619702 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21649f79-fc70-4398-b5ca-f6dc70874c62 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:50.619994 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21649f79-fc70-4398-b5ca-f6dc70874c62 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:50.620271 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21649f79-fc70-4398-b5ca-f6dc70874c62 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:50.620562 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab46ba5-cda3-4241-9706-0361a4ddc42b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:50.621671 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 92/170] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:50.624969 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:50.641176 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0eb8c2-b6cf-40f0-a32d-0f585abb4069 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:50.641525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0eb8c2-b6cf-40f0-a32d-0f585abb4069 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:50.641733 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0eb8c2-b6cf-40f0-a32d-0f585abb4069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:50.653619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:50.673461 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5f0eb8c2-b6cf-40f0-a32d-0f585abb4069 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:50.675906 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5f0eb8c2-b6cf-40f0-a32d-0f585abb4069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:50.709294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab46ba5-cda3-4241-9706-0361a4ddc42b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:50.711365 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab46ba5-cda3-4241-9706-0361a4ddc42b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:50.832838 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 79/171] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:50.841666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab46ba5-cda3-4241-9706-0361a4ddc42b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:50.864462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 93/172] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:50.896534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:50.899548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0eb8c2-b6cf-40f0-a32d-0f585abb4069 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:51.022150 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab46ba5-cda3-4241-9706-0361a4ddc42b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:51.068821 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f4032-1a49-4f5b-9bef-dda8f20d0676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:51.069705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f4032-1a49-4f5b-9bef-dda8f20d0676 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.069985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f4032-1a49-4f5b-9bef-dda8f20d0676 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.070253 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f4032-1a49-4f5b-9bef-dda8f20d0676 None admin] PATH_INFO: `/v3/projects/0e99c982fb1c4cce8a268f9ce2c3383f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.072498 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f4032-1a49-4f5b-9bef-dda8f20d0676 None admin] RBAC: Authorizing `identity:get_project(project_id=0e99c982fb1c4cce8a268f9ce2c3383f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:51.073957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f4032-1a49-4f5b-9bef-dda8f20d0676 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:51.083745 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 80/173] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:50 2022] GET /identity/v3/projects/0e99c982fb1c4cce8a268f9ce2c3383f => generated 312 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:51.084942 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f4032-1a49-4f5b-9bef-dda8f20d0676 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:51.145411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0eb8c2-b6cf-40f0-a32d-0f585abb4069 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:51.224921 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3f16e-6794-4144-be49-30121c207635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:51.226145 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3f16e-6794-4144-be49-30121c207635 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.226622 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3f16e-6794-4144-be49-30121c207635 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.227109 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3f16e-6794-4144-be49-30121c207635 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.235950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:51.247096 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3f16e-6794-4144-be49-30121c207635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:51.248116 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f4032-1a49-4f5b-9bef-dda8f20d0676 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:51.253170 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:51.260669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-58a3f16e-6794-4144-be49-30121c207635 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:51.260669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:51.260669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:51.260669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:51.260669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:51.260669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:51.260669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:51.260669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:51.260669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:51.260669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:51.260669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:51.260669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:51.260669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:51.260669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:51.260669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:51.260669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:51.261730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:51.261730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:51.261730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:51.261730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:51.261730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:51.261730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:51.261730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:51.261730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:51.261730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:51.261730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:51.261730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:51.261730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:51.261730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:51.261730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:51.261730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:51.261730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:51.262586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:51.262586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:51.262586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:51.262586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:51.262586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:51.262586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:51.262586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:51.262586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:51.262586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:51.262586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:51.262586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:51.262586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:02:51.265029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 94/174] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:02:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:51.266545 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3a3de-8ad0-4144-9a1e-5387ddf2e96b None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.266951 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3a3de-8ad0-4144-9a1e-5387ddf2e96b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.267362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3a3de-8ad0-4144-9a1e-5387ddf2e96b None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.268838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 95/175] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:51.270179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02186618-9dcf-4813-b1f5-974127c2cc55 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.270658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02186618-9dcf-4813-b1f5-974127c2cc55 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.271118 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02186618-9dcf-4813-b1f5-974127c2cc55 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.273235 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 96/176] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:51.276485 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02186618-9dcf-4813-b1f5-974127c2cc55 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:51.304014 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765563c3-ef54-4e59-b401-2cacd9691d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:51.304819 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765563c3-ef54-4e59-b401-2cacd9691d4e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.305219 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765563c3-ef54-4e59-b401-2cacd9691d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.305591 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765563c3-ef54-4e59-b401-2cacd9691d4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.306523 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765563c3-ef54-4e59-b401-2cacd9691d4e None admin] RBAC: Adding query filter params (role.id=b1bcb7445ee54636ae7b29a2485cc201, scope.project.id=0e99c982fb1c4cce8a268f9ce2c3383f, user.id=a393cbda710c4f2ca8a06ed49b1c5e5b) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:51.306932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765563c3-ef54-4e59-b401-2cacd9691d4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:51.308494 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765563c3-ef54-4e59-b401-2cacd9691d4e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:51.319733 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 81/177] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:51 2022] GET /identity/v3/role_assignments?user.id=a393cbda710c4f2ca8a06ed49b1c5e5b&scope.project.id=0e99c982fb1c4cce8a268f9ce2c3383f&role.id=b1bcb7445ee54636ae7b29a2485cc201 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:51.321290 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864f387-f822-4c44-8228-0ffdf5121c55 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.321686 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864f387-f822-4c44-8228-0ffdf5121c55 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.322056 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864f387-f822-4c44-8228-0ffdf5121c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.348227 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1864f387-f822-4c44-8228-0ffdf5121c55 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:51.349531 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1864f387-f822-4c44-8228-0ffdf5121c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:51.402173 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02186618-9dcf-4813-b1f5-974127c2cc55 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:51.426820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 82/178] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:51.428316 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f132cb-fdc0-4985-8adc-d326dbf43176 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.429071 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f132cb-fdc0-4985-8adc-d326dbf43176 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.429071 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f132cb-fdc0-4985-8adc-d326dbf43176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.445798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38913f23-9827-410f-9b8b-44c3bced13bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:51.446814 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38913f23-9827-410f-9b8b-44c3bced13bf None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.446938 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a9f132cb-fdc0-4985-8adc-d326dbf43176 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:51.447356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38913f23-9827-410f-9b8b-44c3bced13bf None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.447823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38913f23-9827-410f-9b8b-44c3bced13bf None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.448485 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a9f132cb-fdc0-4985-8adc-d326dbf43176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:51.449047 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38913f23-9827-410f-9b8b-44c3bced13bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:51.449500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38913f23-9827-410f-9b8b-44c3bced13bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:51.451134 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38913f23-9827-410f-9b8b-44c3bced13bf None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:51.461526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 97/179] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:02:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:51.463657 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c7221-f137-467b-a560-121d2eedb88d None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.468143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c7221-f137-467b-a560-121d2eedb88d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.468429 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c7221-f137-467b-a560-121d2eedb88d None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.470301 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 98/180] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:51.471278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c7221-f137-467b-a560-121d2eedb88d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:51.523181 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 83/181] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:51.528525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f132cb-fdc0-4985-8adc-d326dbf43176 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:51.537538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:51.564292 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:51.590772 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c7221-f137-467b-a560-121d2eedb88d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:51.631131 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f24ae-ec5e-4d0b-ba5a-afe3da16e2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:51.632296 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f24ae-ec5e-4d0b-ba5a-afe3da16e2de None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.632851 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f24ae-ec5e-4d0b-ba5a-afe3da16e2de None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.633080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f24ae-ec5e-4d0b-ba5a-afe3da16e2de None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.634261 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f24ae-ec5e-4d0b-ba5a-afe3da16e2de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:51.635777 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f24ae-ec5e-4d0b-ba5a-afe3da16e2de None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:51.637587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 99/182] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:51.638611 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6de5a-8777-4627-bf60-7003b63c309f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.638960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6de5a-8777-4627-bf60-7003b63c309f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.639237 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6de5a-8777-4627-bf60-7003b63c309f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.655343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f132cb-fdc0-4985-8adc-d326dbf43176 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:51.655492 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9bd6de5a-8777-4627-bf60-7003b63c309f None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:51.656623 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9bd6de5a-8777-4627-bf60-7003b63c309f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:51.698508 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5b559-3960-4a44-b86e-ea7b9574ea7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:51.699394 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5b559-3960-4a44-b86e-ea7b9574ea7b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.699700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5b559-3960-4a44-b86e-ea7b9574ea7b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.699974 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5b559-3960-4a44-b86e-ea7b9574ea7b None admin] PATH_INFO: `/v3/users/a393cbda710c4f2ca8a06ed49b1c5e5b` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.701191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5b559-3960-4a44-b86e-ea7b9574ea7b None admin] RBAC: Authorizing `identity:get_user(user_id=a393cbda710c4f2ca8a06ed49b1c5e5b)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:51.702658 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5b559-3960-4a44-b86e-ea7b9574ea7b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:51.704490 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 84/183] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:51 2022] GET /identity/v3/users/a393cbda710c4f2ca8a06ed49b1c5e5b => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:51.705158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5b559-3960-4a44-b86e-ea7b9574ea7b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:51.726574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 100/184] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:51.727455 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6de5a-8777-4627-bf60-7003b63c309f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:51.784070 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:51.822252 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:51.827095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6de5a-8777-4627-bf60-7003b63c309f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:51.830062 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5b559-3960-4a44-b86e-ea7b9574ea7b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:51.860032 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7485eca-0bbd-4cb3-9c7e-9a150d0d2465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:51.860712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7485eca-0bbd-4cb3-9c7e-9a150d0d2465 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.860988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7485eca-0bbd-4cb3-9c7e-9a150d0d2465 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.861260 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7485eca-0bbd-4cb3-9c7e-9a150d0d2465 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.862044 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7485eca-0bbd-4cb3-9c7e-9a150d0d2465 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:51.863522 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7485eca-0bbd-4cb3-9c7e-9a150d0d2465 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:51.866913 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0635a6-ef65-4a70-837e-681578797fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:51.867908 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0635a6-ef65-4a70-837e-681578797fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.868329 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0635a6-ef65-4a70-837e-681578797fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.868741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0635a6-ef65-4a70-837e-681578797fd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.881669 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0635a6-ef65-4a70-837e-681578797fd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:51.885632 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 101/185] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Jun 27 21:02:51 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:02:51.886591 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7485eca-0bbd-4cb3-9c7e-9a150d0d2465 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:51.890582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-8f0635a6-ef65-4a70-837e-681578797fd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:51.890582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:51.890582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:51.890582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:51.890582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:51.890582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:51.890582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:51.890582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:51.890582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:51.890582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:51.890582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:51.890582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:51.890582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:51.890582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:51.890582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:51.890582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:51.891520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:51.891520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:51.891520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:51.891520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:51.891520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:51.891520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:51.891520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:51.891520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:51.891520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:51.891520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:51.891520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:51.891520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:51.891520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:51.891520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:51.891520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:51.891520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:51.892324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:51.892324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:51.892324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:51.892324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:51.892324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:51.892324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:51.892324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:51.892324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:51.892324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:51.892324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:51.892324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:51.892324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:02:51.892324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 85/186] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:02:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:51.893046 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0635a6-ef65-4a70-837e-681578797fd9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:52.020462 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0635a6-ef65-4a70-837e-681578797fd9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:52.060348 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7485eca-0bbd-4cb3-9c7e-9a150d0d2465 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:52.061731 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2fd88-0f29-4497-b021-b7f47c1479d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:52.062786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2fd88-0f29-4497-b021-b7f47c1479d9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:52.063215 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2fd88-0f29-4497-b021-b7f47c1479d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:52.063687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2fd88-0f29-4497-b021-b7f47c1479d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:52.073457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:52.076966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2fd88-0f29-4497-b021-b7f47c1479d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:52.086619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-b2c2fd88-0f29-4497-b021-b7f47c1479d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:52.086619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:52.086619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:52.086619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:52.086619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:52.086619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:52.086619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:52.086619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:52.086619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:52.086619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:52.086619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:52.086619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:52.086619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:52.086619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:52.086619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:52.086619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:52.087626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:52.087626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:52.087626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:52.087626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:52.087626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:52.087626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:52.087626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:52.087626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:52.087626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:52.087626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:52.087626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:52.087626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:52.087626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:52.087626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:52.087626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:52.087626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:52.088530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:52.088530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:52.088530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:52.088530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:52.088530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:52.088530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:52.088530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:52.088530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:52.088530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:52.088530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:52.088530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:52.088530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:02:52.088530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 86/187] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:02:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:52.089157 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2fd88-0f29-4497-b021-b7f47c1479d9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:52.111384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0529e5-db3d-4600-ae6c-4600268ce466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:52.112172 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:52.112279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0529e5-db3d-4600-ae6c-4600268ce466 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:52.112524 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0529e5-db3d-4600-ae6c-4600268ce466 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:52.112803 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0529e5-db3d-4600-ae6c-4600268ce466 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:52.210800 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0529e5-db3d-4600-ae6c-4600268ce466 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:52.213420 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0529e5-db3d-4600-ae6c-4600268ce466 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:52.221963 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2fd88-0f29-4497-b021-b7f47c1479d9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:52.272969 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcad18d-c391-4908-8349-2f57c7f5c36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:52.273645 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcad18d-c391-4908-8349-2f57c7f5c36a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:52.274071 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcad18d-c391-4908-8349-2f57c7f5c36a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:52.274363 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcad18d-c391-4908-8349-2f57c7f5c36a None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:52.275124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcad18d-c391-4908-8349-2f57c7f5c36a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:52.275450 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcad18d-c391-4908-8349-2f57c7f5c36a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:52.277178 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcad18d-c391-4908-8349-2f57c7f5c36a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:52.290217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 87/188] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:52.291409 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcad18d-c391-4908-8349-2f57c7f5c36a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:52.345433 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 102/189] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:52.346319 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0529e5-db3d-4600-ae6c-4600268ce466 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:52.358838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:52.386292 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:52.409357 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcad18d-c391-4908-8349-2f57c7f5c36a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:52.444113 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1602f5-56e5-42f6-adc8-e3309f5cca23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:52.445161 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1602f5-56e5-42f6-adc8-e3309f5cca23 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:52.445673 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1602f5-56e5-42f6-adc8-e3309f5cca23 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:52.446139 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1602f5-56e5-42f6-adc8-e3309f5cca23 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:52.447085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1602f5-56e5-42f6-adc8-e3309f5cca23 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:52.447622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1602f5-56e5-42f6-adc8-e3309f5cca23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:52.449233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1602f5-56e5-42f6-adc8-e3309f5cca23 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:52.450389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0529e5-db3d-4600-ae6c-4600268ce466 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:52.460203 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 88/190] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:02:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:52.460886 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1602f5-56e5-42f6-adc8-e3309f5cca23 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:52.490633 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07e9c7-509e-4cd7-ab4a-b2bf9d0b4fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:52.491638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07e9c7-509e-4cd7-ab4a-b2bf9d0b4fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:52.491936 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07e9c7-509e-4cd7-ab4a-b2bf9d0b4fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:52.492210 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07e9c7-509e-4cd7-ab4a-b2bf9d0b4fd7 None admin] PATH_INFO: `/v3/users/f5bb1f06e8d942729c697de95e5b0448` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:52.507347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a07e9c7-509e-4cd7-ab4a-b2bf9d0b4fd7 None admin] RBAC: Authorizing `identity:get_user(user_id=f5bb1f06e8d942729c697de95e5b0448)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:52.508929 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a07e9c7-509e-4cd7-ab4a-b2bf9d0b4fd7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:52.510951 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 103/191] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:52 2022] GET /identity/v3/users/f5bb1f06e8d942729c697de95e5b0448 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:52.516630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67adc14-f9ea-4c3b-ba57-d0c8f8a4ca90 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:52.516919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67adc14-f9ea-4c3b-ba57-d0c8f8a4ca90 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:52.517198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67adc14-f9ea-4c3b-ba57-d0c8f8a4ca90 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:52.518649 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 104/192] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:52.519387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67adc14-f9ea-4c3b-ba57-d0c8f8a4ca90 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:52.596271 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1602f5-56e5-42f6-adc8-e3309f5cca23 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:52.605973 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:52.633684 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cf51e-0fc3-45b0-bd33-67a416f6160e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:52.634570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cf51e-0fc3-45b0-bd33-67a416f6160e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:52.634876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cf51e-0fc3-45b0-bd33-67a416f6160e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:52.635454 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cf51e-0fc3-45b0-bd33-67a416f6160e None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:52.636556 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23cf51e-0fc3-45b0-bd33-67a416f6160e None admin] RBAC: Authorizing `identity:get_project(project_id=0dc9df6fada54f72a35899fa21f03b20)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:52.638041 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23cf51e-0fc3-45b0-bd33-67a416f6160e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:52.640017 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 89/193] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:52 2022] GET /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:52.640831 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cf51e-0fc3-45b0-bd33-67a416f6160e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:52.650488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67adc14-f9ea-4c3b-ba57-d0c8f8a4ca90 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:52.653324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:52.693604 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b72f82-e031-4825-8695-661d77dd93f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:52.695181 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b72f82-e031-4825-8695-661d77dd93f9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:52.695951 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b72f82-e031-4825-8695-661d77dd93f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:52.696570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b72f82-e031-4825-8695-661d77dd93f9 None admin] PATH_INFO: `/v3/users/f5bb1f06e8d942729c697de95e5b0448` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:52.698573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b72f82-e031-4825-8695-661d77dd93f9 None admin] RBAC: Authorizing `identity:get_user(user_id=f5bb1f06e8d942729c697de95e5b0448)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:52.700677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b72f82-e031-4825-8695-661d77dd93f9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:52.707704 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 105/194] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:52 2022] GET /identity/v3/users/f5bb1f06e8d942729c697de95e5b0448 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:52.709494 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c461a6-ff9b-4f6b-9175-d8d04d4bd38d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:52.710087 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c461a6-ff9b-4f6b-9175-d8d04d4bd38d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:52.710862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c461a6-ff9b-4f6b-9175-d8d04d4bd38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:52.731737 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-13c461a6-ff9b-4f6b-9175-d8d04d4bd38d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:52.733850 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-13c461a6-ff9b-4f6b-9175-d8d04d4bd38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:52.760634 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cf51e-0fc3-45b0-bd33-67a416f6160e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:52.804564 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059782e1-aeff-4219-b028-e7a462d251a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:52.805268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059782e1-aeff-4219-b028-e7a462d251a9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:52.805783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059782e1-aeff-4219-b028-e7a462d251a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:52.806074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059782e1-aeff-4219-b028-e7a462d251a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:52.818881 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 106/195] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:52.824779 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c461a6-ff9b-4f6b-9175-d8d04d4bd38d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:52.892430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059782e1-aeff-4219-b028-e7a462d251a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:52.894153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059782e1-aeff-4219-b028-e7a462d251a9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:52.908011 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:52.911865 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:52.948996 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c461a6-ff9b-4f6b-9175-d8d04d4bd38d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:52.984135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018f284-0c6d-48dc-b39f-531df83299a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:52.985125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018f284-0c6d-48dc-b39f-531df83299a3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:52.985594 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018f284-0c6d-48dc-b39f-531df83299a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:52.986086 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018f284-0c6d-48dc-b39f-531df83299a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:52.987141 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a018f284-0c6d-48dc-b39f-531df83299a3 None admin] RBAC: Adding query filter params (role.id=b3efb2b2e43c4af8b1cf36a37f1e05f9, scope.project.id=0dc9df6fada54f72a35899fa21f03b20, user.id=a393cbda710c4f2ca8a06ed49b1c5e5b) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:52.987660 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a018f284-0c6d-48dc-b39f-531df83299a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:52.989406 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a018f284-0c6d-48dc-b39f-531df83299a3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:52.995768 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 107/196] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:52 2022] GET /identity/v3/role_assignments?user.id=a393cbda710c4f2ca8a06ed49b1c5e5b&scope.project.id=0dc9df6fada54f72a35899fa21f03b20&role.id=b3efb2b2e43c4af8b1cf36a37f1e05f9 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:52.997316 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018f284-0c6d-48dc-b39f-531df83299a3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:53.024622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 90/197] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:53.025570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059782e1-aeff-4219-b028-e7a462d251a9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:53.104994 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018f284-0c6d-48dc-b39f-531df83299a3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:53.135871 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:53.142754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defda2b1-f985-431e-827f-a3f063898f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:53.143920 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defda2b1-f985-431e-827f-a3f063898f98 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:53.144384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defda2b1-f985-431e-827f-a3f063898f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:53.149673 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defda2b1-f985-431e-827f-a3f063898f98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:53.206219 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059782e1-aeff-4219-b028-e7a462d251a9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:53.221108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:53.222382 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defda2b1-f985-431e-827f-a3f063898f98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:53.224200 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defda2b1-f985-431e-827f-a3f063898f98 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:53.288468 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae9495-1ff1-49cb-b7a1-a08abf5748b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:53.289614 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae9495-1ff1-49cb-b7a1-a08abf5748b6 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:53.289902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae9495-1ff1-49cb-b7a1-a08abf5748b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:53.290182 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae9495-1ff1-49cb-b7a1-a08abf5748b6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:53.309031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ae9495-1ff1-49cb-b7a1-a08abf5748b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:53.320214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-81ae9495-1ff1-49cb-b7a1-a08abf5748b6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:53.320214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:53.320214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:53.320214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:53.320214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:53.320214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:53.320214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:53.320214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:53.320214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:53.320214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:53.320214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:53.320214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:53.320214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:53.320214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:53.320214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:53.320214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:53.321124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:53.321124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:53.321124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:53.321124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:53.321124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:53.321124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:53.321124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:53.321124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:53.321124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:53.321124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:53.321124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:53.321124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:53.321124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:53.321124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:53.321124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:53.321124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:53.322294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:53.322294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:53.322294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:53.322294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:53.322294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:53.322294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:53.322294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:53.322294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:53.322294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:53.322294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:53.322294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:53.322294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:02:53.322294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 91/198] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:53.322294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae9495-1ff1-49cb-b7a1-a08abf5748b6 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:53.387439 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 108/199] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:53.388334 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defda2b1-f985-431e-827f-a3f063898f98 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:53.429434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:53.462741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae9495-1ff1-49cb-b7a1-a08abf5748b6 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:53.501540 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615425e6-5f73-414c-9124-2b18754c41b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:53.502731 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615425e6-5f73-414c-9124-2b18754c41b3 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:53.503148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615425e6-5f73-414c-9124-2b18754c41b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:53.503555 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615425e6-5f73-414c-9124-2b18754c41b3 None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:53.505872 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615425e6-5f73-414c-9124-2b18754c41b3 None admin] RBAC: Authorizing `identity:get_project(project_id=e7099ea37d874a7aa73b15453887dffc)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:53.507350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615425e6-5f73-414c-9124-2b18754c41b3 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:53.509401 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 92/200] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:53 2022] GET /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:53.510239 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615425e6-5f73-414c-9124-2b18754c41b3 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:53.519844 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defda2b1-f985-431e-827f-a3f063898f98 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:53.552066 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:53.561857 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e823f-9b1c-4255-bc53-071f6c547708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:53.562678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e823f-9b1c-4255-bc53-071f6c547708 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:53.562857 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e823f-9b1c-4255-bc53-071f6c547708 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:53.563085 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e823f-9b1c-4255-bc53-071f6c547708 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:53.563873 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e823f-9b1c-4255-bc53-071f6c547708 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:53.564162 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e823f-9b1c-4255-bc53-071f6c547708 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:53.565747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e823f-9b1c-4255-bc53-071f6c547708 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:53.575095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 109/201] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:53.576023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e823f-9b1c-4255-bc53-071f6c547708 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:53.628209 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615425e6-5f73-414c-9124-2b18754c41b3 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:53.661215 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf510bf8-43f7-4f1a-9782-81d5ad1c21a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:53.662104 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf510bf8-43f7-4f1a-9782-81d5ad1c21a5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:53.662393 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf510bf8-43f7-4f1a-9782-81d5ad1c21a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:53.662675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf510bf8-43f7-4f1a-9782-81d5ad1c21a5 None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:53.663888 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf510bf8-43f7-4f1a-9782-81d5ad1c21a5 None admin] RBAC: Authorizing `identity:get_project(project_id=e7099ea37d874a7aa73b15453887dffc)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:53.665375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf510bf8-43f7-4f1a-9782-81d5ad1c21a5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:53.667395 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 93/202] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:53 2022] GET /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:53.668269 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf510bf8-43f7-4f1a-9782-81d5ad1c21a5 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:53.690374 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:53.703335 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e823f-9b1c-4255-bc53-071f6c547708 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:53.748345 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d455e-8a55-4f12-a2f0-e34d51e3f4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:53.749637 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d455e-8a55-4f12-a2f0-e34d51e3f4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:53.750059 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d455e-8a55-4f12-a2f0-e34d51e3f4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:53.750467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d455e-8a55-4f12-a2f0-e34d51e3f4a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:53.752031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d455e-8a55-4f12-a2f0-e34d51e3f4a0 None admin] RBAC: Adding query filter params (role.id=5c3c095bb53943eca95c4e049cd8cbf5, scope.project.id=e7099ea37d874a7aa73b15453887dffc, user.id=f5bb1f06e8d942729c697de95e5b0448) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:53.752646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d455e-8a55-4f12-a2f0-e34d51e3f4a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:53.754744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d455e-8a55-4f12-a2f0-e34d51e3f4a0 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:53.762109 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 110/203] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:53 2022] GET /identity/v3/role_assignments?user.id=f5bb1f06e8d942729c697de95e5b0448&scope.project.id=e7099ea37d874a7aa73b15453887dffc&role.id=5c3c095bb53943eca95c4e049cd8cbf5 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:53.763291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d455e-8a55-4f12-a2f0-e34d51e3f4a0 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:53.773168 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf510bf8-43f7-4f1a-9782-81d5ad1c21a5 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:53.775854 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:53.812707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121fa3a-a750-43ef-bf6a-70d485c3f50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:53.813676 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121fa3a-a750-43ef-bf6a-70d485c3f50c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:53.814023 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121fa3a-a750-43ef-bf6a-70d485c3f50c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:53.814371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121fa3a-a750-43ef-bf6a-70d485c3f50c None admin] PATH_INFO: `/v3/users/c188f05705be4e49b79393cc268ace22` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:53.815824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121fa3a-a750-43ef-bf6a-70d485c3f50c None admin] RBAC: Authorizing `identity:get_user(user_id=c188f05705be4e49b79393cc268ace22)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:53.817971 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121fa3a-a750-43ef-bf6a-70d485c3f50c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:53.819463 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 94/204] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:53 2022] GET /identity/v3/users/c188f05705be4e49b79393cc268ace22 => generated 289 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:53.824971 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121fa3a-a750-43ef-bf6a-70d485c3f50c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:53.895998 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d455e-8a55-4f12-a2f0-e34d51e3f4a0 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:53.943649 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121fa3a-a750-43ef-bf6a-70d485c3f50c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:53.954373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da918a17-d734-4c7a-b702-97d9a0c37da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:53.955561 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da918a17-d734-4c7a-b702-97d9a0c37da3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:53.956662 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da918a17-d734-4c7a-b702-97d9a0c37da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:53.957131 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da918a17-d734-4c7a-b702-97d9a0c37da3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:53.961748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da918a17-d734-4c7a-b702-97d9a0c37da3 None admin] RBAC: Adding query filter params (role.id=b3efb2b2e43c4af8b1cf36a37f1e05f9, scope.project.id=e7099ea37d874a7aa73b15453887dffc, user.id=f5bb1f06e8d942729c697de95e5b0448) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:53.962346 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da918a17-d734-4c7a-b702-97d9a0c37da3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:53.964144 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da918a17-d734-4c7a-b702-97d9a0c37da3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:53.980319 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 111/205] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:53 2022] GET /identity/v3/role_assignments?user.id=f5bb1f06e8d942729c697de95e5b0448&scope.project.id=e7099ea37d874a7aa73b15453887dffc&role.id=b3efb2b2e43c4af8b1cf36a37f1e05f9 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:54.017787 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf31ee-5c0b-4714-81f8-0de4453f78df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:54.018580 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf31ee-5c0b-4714-81f8-0de4453f78df None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.018873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf31ee-5c0b-4714-81f8-0de4453f78df None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.019154 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf31ee-5c0b-4714-81f8-0de4453f78df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.080133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:54.098003 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf31ee-5c0b-4714-81f8-0de4453f78df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:54.099694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf31ee-5c0b-4714-81f8-0de4453f78df None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:54.211010 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 95/206] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 21:02:54.217011 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da918a17-d734-4c7a-b702-97d9a0c37da3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:54.239702 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:54.341049 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da918a17-d734-4c7a-b702-97d9a0c37da3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:54.383490 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318801ce-e81f-44cd-97c7-b4b6a2cf33ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:54.384418 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318801ce-e81f-44cd-97c7-b4b6a2cf33ee None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.384703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318801ce-e81f-44cd-97c7-b4b6a2cf33ee None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.384975 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318801ce-e81f-44cd-97c7-b4b6a2cf33ee None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.386230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318801ce-e81f-44cd-97c7-b4b6a2cf33ee None admin] RBAC: Authorizing `identity:get_project(project_id=0dc9df6fada54f72a35899fa21f03b20)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:54.387734 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318801ce-e81f-44cd-97c7-b4b6a2cf33ee None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:54.389542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 112/207] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:54 2022] GET /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:54.393880 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d1521-01af-4a5c-8840-0fd8138e90e9 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.394236 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d1521-01af-4a5c-8840-0fd8138e90e9 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.394586 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d1521-01af-4a5c-8840-0fd8138e90e9 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.396003 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 96/208] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:54.396924 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318801ce-e81f-44cd-97c7-b4b6a2cf33ee None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:54.404860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8c53d-0d8d-4094-8a07-0cb6df3ad736 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.405148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8c53d-0d8d-4094-8a07-0cb6df3ad736 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.405428 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8c53d-0d8d-4094-8a07-0cb6df3ad736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.432417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-93d8c53d-0d8d-4094-8a07-0cb6df3ad736 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:54.434347 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-93d8c53d-0d8d-4094-8a07-0cb6df3ad736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:54.524566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 97/209] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:54.527476 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56578134-86e7-4614-880c-67239ae3c56a None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.527787 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56578134-86e7-4614-880c-67239ae3c56a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.528064 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56578134-86e7-4614-880c-67239ae3c56a None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.529374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 98/210] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:54.534972 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc24c30-d732-441f-8da2-7dd5f14df259 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.535254 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc24c30-d732-441f-8da2-7dd5f14df259 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.535827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc24c30-d732-441f-8da2-7dd5f14df259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.542108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:54.555488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:54.565620 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-7fc24c30-d732-441f-8da2-7dd5f14df259 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:54.567384 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7fc24c30-d732-441f-8da2-7dd5f14df259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:54.571089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318801ce-e81f-44cd-97c7-b4b6a2cf33ee None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:54.612089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3ae4b-db48-4dab-9906-9991fb30e771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:54.612731 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3ae4b-db48-4dab-9906-9991fb30e771 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.613016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3ae4b-db48-4dab-9906-9991fb30e771 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.613293 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3ae4b-db48-4dab-9906-9991fb30e771 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.614270 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3ae4b-db48-4dab-9906-9991fb30e771 None admin] RBAC: Adding query filter params (role.id=e71d790e22744442b6ff4af3caa1e853, scope.project.id=0dc9df6fada54f72a35899fa21f03b20, user.id=c188f05705be4e49b79393cc268ace22) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:54.614686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3ae4b-db48-4dab-9906-9991fb30e771 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:54.616233 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3ae4b-db48-4dab-9906-9991fb30e771 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:54.623014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 113/211] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:54 2022] GET /identity/v3/role_assignments?user.id=c188f05705be4e49b79393cc268ace22&scope.project.id=0dc9df6fada54f72a35899fa21f03b20&role.id=e71d790e22744442b6ff4af3caa1e853 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:54.623963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3ae4b-db48-4dab-9906-9991fb30e771 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:54.656598 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 99/212] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:54.684084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc24c30-d732-441f-8da2-7dd5f14df259 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:54.824599 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3ae4b-db48-4dab-9906-9991fb30e771 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:54.849423 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc24c30-d732-441f-8da2-7dd5f14df259 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:54.861915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ee22f-c534-48da-b559-25244772d7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:54.862788 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ee22f-c534-48da-b559-25244772d7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.863010 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ee22f-c534-48da-b559-25244772d7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.863283 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ee22f-c534-48da-b559-25244772d7b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.864687 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ee22f-c534-48da-b559-25244772d7b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:54.866185 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ee22f-c534-48da-b559-25244772d7b7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:54.868064 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 114/213] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:54.874848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ee22f-c534-48da-b559-25244772d7b7 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:54.880658 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:54.885408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf26e8f-e745-4f9e-9269-0c0bd22035f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:54.886283 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf26e8f-e745-4f9e-9269-0c0bd22035f2 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.886565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf26e8f-e745-4f9e-9269-0c0bd22035f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.886832 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf26e8f-e745-4f9e-9269-0c0bd22035f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.897903 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf26e8f-e745-4f9e-9269-0c0bd22035f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:54.901795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:54.906078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-aaf26e8f-e745-4f9e-9269-0c0bd22035f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:54.906078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:54.906078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:54.906078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:54.906078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:54.906078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:54.906078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:54.906078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:54.906078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:54.906078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:54.906078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:54.906078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:54.906078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:54.906078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:54.906078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:54.906078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:54.908846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:54.908846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:54.908846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:54.908846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:54.908846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:54.908846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:54.908846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:54.908846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:54.908846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:54.908846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:54.908846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:54.908846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:54.908846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:54.908846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:54.908846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:54.908846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:54.910751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:54.910751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:54.910751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:54.910751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:54.910751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:54.910751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:54.910751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:54.910751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:54.910751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:54.910751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:54.910751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:54.910751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:02:54.910751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 100/214] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:54.916825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf26e8f-e745-4f9e-9269-0c0bd22035f2 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:55.000328 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ee22f-c534-48da-b559-25244772d7b7 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:55.038964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdac42a-f130-42c0-bb20-dad8f577f33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:55.039786 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdac42a-f130-42c0-bb20-dad8f577f33a None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:55.040070 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdac42a-f130-42c0-bb20-dad8f577f33a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:55.040345 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdac42a-f130-42c0-bb20-dad8f577f33a None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:55.041064 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdac42a-f130-42c0-bb20-dad8f577f33a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:55.042305 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf26e8f-e745-4f9e-9269-0c0bd22035f2 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:55.042542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdac42a-f130-42c0-bb20-dad8f577f33a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:55.067101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 115/215] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Jun 27 21:02:54 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:02:55.079769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab163419-8d95-41df-b5a4-bbcd8990cb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:55.080534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab163419-8d95-41df-b5a4-bbcd8990cb8c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:55.081124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab163419-8d95-41df-b5a4-bbcd8990cb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:55.081124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab163419-8d95-41df-b5a4-bbcd8990cb8c None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:55.081841 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab163419-8d95-41df-b5a4-bbcd8990cb8c None admin] RBAC: Adding query filter params (name=member) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:55.082151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab163419-8d95-41df-b5a4-bbcd8990cb8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:55.083642 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab163419-8d95-41df-b5a4-bbcd8990cb8c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:55.094545 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 101/216] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:55.100828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdac42a-f130-42c0-bb20-dad8f577f33a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:55.202316 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:55.232730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdac42a-f130-42c0-bb20-dad8f577f33a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:55.270569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9368613-adf2-4f01-a6eb-806071644d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:55.271475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9368613-adf2-4f01-a6eb-806071644d9b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:55.271781 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9368613-adf2-4f01-a6eb-806071644d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:55.272055 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9368613-adf2-4f01-a6eb-806071644d9b None admin] PATH_INFO: `/v3/users/5864827f36c54aa89f4bc422ba9d8874` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:55.286701 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9368613-adf2-4f01-a6eb-806071644d9b None admin] RBAC: Authorizing `identity:get_user(user_id=5864827f36c54aa89f4bc422ba9d8874)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:55.288274 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9368613-adf2-4f01-a6eb-806071644d9b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:55.290414 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 116/217] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:55 2022] GET /identity/v3/users/5864827f36c54aa89f4bc422ba9d8874 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:55.295929 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab163419-8d95-41df-b5a4-bbcd8990cb8c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:55.359828 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:55.404437 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab163419-8d95-41df-b5a4-bbcd8990cb8c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:55.444488 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ddb73-4851-4ef5-9b6c-b2d25418dc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:55.445150 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ddb73-4851-4ef5-9b6c-b2d25418dc76 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:55.445426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ddb73-4851-4ef5-9b6c-b2d25418dc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:55.445697 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ddb73-4851-4ef5-9b6c-b2d25418dc76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:55.524490 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ddb73-4851-4ef5-9b6c-b2d25418dc76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:55.525965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ddb73-4851-4ef5-9b6c-b2d25418dc76 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:55.583257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:55.632569 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 102/218] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 21:02:55.638449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9368613-adf2-4f01-a6eb-806071644d9b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:55.771437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9368613-adf2-4f01-a6eb-806071644d9b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:55.818733 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ad213-4662-4b64-ae1f-e7496392421b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:55.819934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ad213-4662-4b64-ae1f-e7496392421b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:55.820382 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ad213-4662-4b64-ae1f-e7496392421b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:55.820819 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ad213-4662-4b64-ae1f-e7496392421b None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:55.822247 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ad213-4662-4b64-ae1f-e7496392421b None admin] RBAC: Authorizing `identity:get_project(project_id=e7099ea37d874a7aa73b15453887dffc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:55.823962 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ad213-4662-4b64-ae1f-e7496392421b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:55.825882 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 117/219] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:55 2022] GET /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:55.832096 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ddb73-4851-4ef5-9b6c-b2d25418dc76 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:56.067643 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ddb73-4851-4ef5-9b6c-b2d25418dc76 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:56.123640 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713ed8d-1681-4dd3-b6e5-dcd7cebba9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:56.124270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713ed8d-1681-4dd3-b6e5-dcd7cebba9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:56.124510 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713ed8d-1681-4dd3-b6e5-dcd7cebba9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:56.124786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713ed8d-1681-4dd3-b6e5-dcd7cebba9b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:56.125660 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713ed8d-1681-4dd3-b6e5-dcd7cebba9b7 None admin] RBAC: Adding query filter params (role.id=b1bcb7445ee54636ae7b29a2485cc201, scope.project.id=e7099ea37d874a7aa73b15453887dffc, user.id=5864827f36c54aa89f4bc422ba9d8874) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:56.125973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713ed8d-1681-4dd3-b6e5-dcd7cebba9b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:56.127461 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713ed8d-1681-4dd3-b6e5-dcd7cebba9b7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:56.134278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 103/220] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:55 2022] GET /identity/v3/role_assignments?user.id=5864827f36c54aa89f4bc422ba9d8874&scope.project.id=e7099ea37d874a7aa73b15453887dffc&role.id=b1bcb7445ee54636ae7b29a2485cc201 => generated 259 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:56.231201 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5681b-b346-4b0c-9d8b-73366ef24328 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:56.231681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5681b-b346-4b0c-9d8b-73366ef24328 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:56.232056 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5681b-b346-4b0c-9d8b-73366ef24328 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:56.233773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 118/221] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:56.239683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ae3c6-22c6-45c9-bec5-d63d17c9a3cc None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:56.239973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ae3c6-22c6-45c9-bec5-d63d17c9a3cc None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:56.240246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ae3c6-22c6-45c9-bec5-d63d17c9a3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:56.273281 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e44ae3c6-22c6-45c9-bec5-d63d17c9a3cc None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:56.274524 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e44ae3c6-22c6-45c9-bec5-d63d17c9a3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:56.293129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:56.401305 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43644a-2d9c-4aa8-b4bf-1f42777148a5 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:56.401476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43644a-2d9c-4aa8-b4bf-1f42777148a5 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:56.405640 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43644a-2d9c-4aa8-b4bf-1f42777148a5 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:56.405640 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 119/222] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:56.405945 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 104/223] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:56.410724 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247cdd1-2668-4b99-8307-18f2fce0a932 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:56.411655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247cdd1-2668-4b99-8307-18f2fce0a932 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:56.411655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247cdd1-2668-4b99-8307-18f2fce0a932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:56.429529 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ae3c6-22c6-45c9-bec5-d63d17c9a3cc None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:56.431964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:56.455759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7247cdd1-2668-4b99-8307-18f2fce0a932 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:56.458510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7247cdd1-2668-4b99-8307-18f2fce0a932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:56.635440 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 120/224] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:56.659673 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ae3c6-22c6-45c9-bec5-d63d17c9a3cc None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:56.661695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247cdd1-2668-4b99-8307-18f2fce0a932 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:56.731060 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4b927-0278-40f0-ba1e-276a0b53dcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:56.731907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4b927-0278-40f0-ba1e-276a0b53dcc2 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:56.732186 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4b927-0278-40f0-ba1e-276a0b53dcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:56.732460 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4b927-0278-40f0-ba1e-276a0b53dcc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:56.751022 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4b927-0278-40f0-ba1e-276a0b53dcc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:56.758868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-03b4b927-0278-40f0-ba1e-276a0b53dcc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:56.758868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:56.758868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:56.758868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:56.758868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:56.758868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:56.758868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:56.758868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:56.758868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:56.758868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:56.758868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:56.758868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:56.758868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:56.758868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:56.758868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:56.758868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:56.759759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:56.759759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:56.759759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:56.759759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:56.759759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:56.759759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:56.759759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:56.759759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:56.759759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:56.759759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:56.759759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:56.759759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:56.759759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:56.759759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:56.759759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:56.759759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:56.760544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:56.760544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:56.760544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:56.760544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:56.760544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:56.760544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:56.760544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:56.760544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:56.760544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:56.760544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:56.760544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:56.760544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:02:56.766185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 105/225] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:02:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:56.776713 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4b927-0278-40f0-ba1e-276a0b53dcc2 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:56.793223 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:56.837969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247cdd1-2668-4b99-8307-18f2fce0a932 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:56.890162 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:56.900959 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451f8de-63f4-41e0-b92b-7cc72e4dff11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:56.902004 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451f8de-63f4-41e0-b92b-7cc72e4dff11 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:56.905790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451f8de-63f4-41e0-b92b-7cc72e4dff11 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:56.906289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451f8de-63f4-41e0-b92b-7cc72e4dff11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:56.924571 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451f8de-63f4-41e0-b92b-7cc72e4dff11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:56.938298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4451f8de-63f4-41e0-b92b-7cc72e4dff11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:56.938298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:56.938298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:56.938298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:56.938298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:56.938298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:56.938298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:56.938298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:56.938298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:56.938298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:56.938298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:56.938298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:56.938298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:56.938298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:56.938298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:56.938298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:56.939362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:56.939362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:56.939362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:56.939362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:56.939362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:56.939362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:56.939362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:56.939362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:56.939362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:56.939362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:56.939362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:56.939362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:56.939362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:56.939362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:56.939362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:56.939362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:56.940411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:56.940411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:56.940411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:56.940411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:56.940411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:56.940411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:56.940411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:56.940411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:56.940411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:56.940411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:56.940411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:56.940411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:02:56.941376 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 121/226] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:02:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:56.946655 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4b927-0278-40f0-ba1e-276a0b53dcc2 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:56.947066 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451f8de-63f4-41e0-b92b-7cc72e4dff11 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:57.008546 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23482084-3fa1-4510-b147-2ad9e167e910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:57.009560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23482084-3fa1-4510-b147-2ad9e167e910 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:57.010036 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23482084-3fa1-4510-b147-2ad9e167e910 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:57.010526 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23482084-3fa1-4510-b147-2ad9e167e910 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:57.011506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23482084-3fa1-4510-b147-2ad9e167e910 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:57.012001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23482084-3fa1-4510-b147-2ad9e167e910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:57.013577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23482084-3fa1-4510-b147-2ad9e167e910 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:57.036570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 106/227] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:57.038155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23482084-3fa1-4510-b147-2ad9e167e910 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:57.152935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451f8de-63f4-41e0-b92b-7cc72e4dff11 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:57.155944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:57.196739 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23482084-3fa1-4510-b147-2ad9e167e910 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:57.209503 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350e3d4-b9c4-4095-9629-e961e2ac9b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:57.210626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350e3d4-b9c4-4095-9629-e961e2ac9b2a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:57.210918 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350e3d4-b9c4-4095-9629-e961e2ac9b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:57.211190 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350e3d4-b9c4-4095-9629-e961e2ac9b2a None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:57.211982 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c350e3d4-b9c4-4095-9629-e961e2ac9b2a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:57.212490 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c350e3d4-b9c4-4095-9629-e961e2ac9b2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:57.214531 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c350e3d4-b9c4-4095-9629-e961e2ac9b2a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:57.231080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 122/228] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:02:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:57.237713 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350e3d4-b9c4-4095-9629-e961e2ac9b2a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:57.240073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb417a-ad90-43ca-b2f2-5018f5e100b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:57.240971 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb417a-ad90-43ca-b2f2-5018f5e100b3 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:57.241254 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb417a-ad90-43ca-b2f2-5018f5e100b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:57.241528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb417a-ad90-43ca-b2f2-5018f5e100b3 None admin] PATH_INFO: `/v3/users/f5bb1f06e8d942729c697de95e5b0448` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:57.242735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb417a-ad90-43ca-b2f2-5018f5e100b3 None admin] RBAC: Authorizing `identity:get_user(user_id=f5bb1f06e8d942729c697de95e5b0448)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:57.244401 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb417a-ad90-43ca-b2f2-5018f5e100b3 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:57.246295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 107/229] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:57 2022] GET /identity/v3/users/f5bb1f06e8d942729c697de95e5b0448 => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:57.252250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb417a-ad90-43ca-b2f2-5018f5e100b3 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:57.344904 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:57.407436 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb417a-ad90-43ca-b2f2-5018f5e100b3 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:57.445016 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128812c-ab10-423b-a807-1e4498ad8f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:57.445893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128812c-ab10-423b-a807-1e4498ad8f54 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:57.446170 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128812c-ab10-423b-a807-1e4498ad8f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:57.446443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128812c-ab10-423b-a807-1e4498ad8f54 None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:57.447890 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a128812c-ab10-423b-a807-1e4498ad8f54 None admin] RBAC: Authorizing `identity:get_project(project_id=e7099ea37d874a7aa73b15453887dffc)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:57.449288 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a128812c-ab10-423b-a807-1e4498ad8f54 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:57.453367 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 108/230] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:57 2022] GET /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:57.456395 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350e3d4-b9c4-4095-9629-e961e2ac9b2a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:57.458483 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128812c-ab10-423b-a807-1e4498ad8f54 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:57.477586 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:57.524967 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a685bc-aa44-4777-a4b4-e3c1b4c43f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:57.525868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a685bc-aa44-4777-a4b4-e3c1b4c43f81 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:57.526144 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a685bc-aa44-4777-a4b4-e3c1b4c43f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:57.526408 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a685bc-aa44-4777-a4b4-e3c1b4c43f81 None admin] PATH_INFO: `/v3/users/f5bb1f06e8d942729c697de95e5b0448` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:57.533096 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a685bc-aa44-4777-a4b4-e3c1b4c43f81 None admin] RBAC: Authorizing `identity:get_user(user_id=f5bb1f06e8d942729c697de95e5b0448)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:57.534673 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a685bc-aa44-4777-a4b4-e3c1b4c43f81 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:57.536698 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 123/231] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:57 2022] GET /identity/v3/users/f5bb1f06e8d942729c697de95e5b0448 => generated 283 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:57.546759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a685bc-aa44-4777-a4b4-e3c1b4c43f81 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:57.646884 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128812c-ab10-423b-a807-1e4498ad8f54 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:57.683131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d86be0-7bd6-45cf-9c36-47c87d85dd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:57.684179 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d86be0-7bd6-45cf-9c36-47c87d85dd9e None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:57.684489 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d86be0-7bd6-45cf-9c36-47c87d85dd9e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:57.684860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d86be0-7bd6-45cf-9c36-47c87d85dd9e None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc/users/f5bb1f06e8d942729c697de95e5b0448/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:57.687946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d86be0-7bd6-45cf-9c36-47c87d85dd9e None admin] RBAC: Authorizing `identity:create_grant(project_id=e7099ea37d874a7aa73b15453887dffc, user_id=f5bb1f06e8d942729c697de95e5b0448, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:57.689796 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d86be0-7bd6-45cf-9c36-47c87d85dd9e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:57.705238 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 109/232] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:02:57 2022] PUT /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc/users/f5bb1f06e8d942729c697de95e5b0448/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:02:57.707039 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a8a04-3743-42ed-8019-08cec7cd83a0 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:57.707330 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a8a04-3743-42ed-8019-08cec7cd83a0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:57.707669 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a8a04-3743-42ed-8019-08cec7cd83a0 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:57.709125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 110/233] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:57.715267 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179204ac-f38a-4fd5-b8a6-e55a2802a2af None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:57.715648 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179204ac-f38a-4fd5-b8a6-e55a2802a2af None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:57.715991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179204ac-f38a-4fd5-b8a6-e55a2802a2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:57.725314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a685bc-aa44-4777-a4b4-e3c1b4c43f81 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:57.735124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-179204ac-f38a-4fd5-b8a6-e55a2802a2af None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:57.736574 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-179204ac-f38a-4fd5-b8a6-e55a2802a2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:57.751838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:57.778636 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28663992-9b4b-4296-8d67-99ab48bff971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:57.779938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28663992-9b4b-4296-8d67-99ab48bff971 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:57.780047 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28663992-9b4b-4296-8d67-99ab48bff971 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:57.780396 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28663992-9b4b-4296-8d67-99ab48bff971 None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:57.783766 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28663992-9b4b-4296-8d67-99ab48bff971 None admin] RBAC: Authorizing `identity:get_project(project_id=e7099ea37d874a7aa73b15453887dffc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:57.785687 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28663992-9b4b-4296-8d67-99ab48bff971 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:57.787273 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 124/234] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:57 2022] GET /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:57.793397 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28663992-9b4b-4296-8d67-99ab48bff971 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:57.812090 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:57.817787 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 111/235] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:57.842579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179204ac-f38a-4fd5-b8a6-e55a2802a2af None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:57.977334 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28663992-9b4b-4296-8d67-99ab48bff971 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:58.005986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179204ac-f38a-4fd5-b8a6-e55a2802a2af None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:58.019978 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fcd5d-a535-4506-9cf8-820bb538901f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:58.021309 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fcd5d-a535-4506-9cf8-820bb538901f None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:58.021730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fcd5d-a535-4506-9cf8-820bb538901f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:58.022145 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fcd5d-a535-4506-9cf8-820bb538901f None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc/users/f5bb1f06e8d942729c697de95e5b0448/roles/b3efb2b2e43c4af8b1cf36a37f1e05f9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:58.025421 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fcd5d-a535-4506-9cf8-820bb538901f None admin] RBAC: Authorizing `identity:create_grant(project_id=e7099ea37d874a7aa73b15453887dffc, user_id=f5bb1f06e8d942729c697de95e5b0448, role_id=b3efb2b2e43c4af8b1cf36a37f1e05f9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:58.027373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fcd5d-a535-4506-9cf8-820bb538901f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:58.045845 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 125/236] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:02:57 2022] PUT /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc/users/f5bb1f06e8d942729c697de95e5b0448/roles/b3efb2b2e43c4af8b1cf36a37f1e05f9 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:02:58.047301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35be667-917e-4c85-83fa-7c54b61c887c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:58.047536 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251166e5-9feb-4392-af19-aafe8bf977d5 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:58.047995 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251166e5-9feb-4392-af19-aafe8bf977d5 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:58.048208 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35be667-917e-4c85-83fa-7c54b61c887c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:58.048502 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35be667-917e-4c85-83fa-7c54b61c887c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:58.048726 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251166e5-9feb-4392-af19-aafe8bf977d5 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:58.048829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35be667-917e-4c85-83fa-7c54b61c887c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:58.050692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 126/237] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:58.056472 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5ead5-2e7e-4677-9968-b93f92d9c8e0 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:58.057003 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5ead5-2e7e-4677-9968-b93f92d9c8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:58.057429 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5ead5-2e7e-4677-9968-b93f92d9c8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:58.062060 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35be667-917e-4c85-83fa-7c54b61c887c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:58.071257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-b35be667-917e-4c85-83fa-7c54b61c887c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:58.071257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:58.071257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:58.071257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:58.071257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:58.071257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:58.071257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:58.071257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:58.071257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:58.071257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:58.071257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:58.071257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:58.071257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:58.071257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:58.071257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:58.071257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:58.072421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:58.072421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:58.072421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:58.072421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:58.072421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:58.072421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:58.072421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:58.072421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:58.072421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:58.072421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:58.072421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:58.072421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:58.072421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:58.072421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:58.072421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:58.072421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:58.073405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:58.073405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:58.073405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:58.073405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:58.073405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:58.073405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:58.073405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:58.073405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:58.073405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:58.073405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:58.073405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:58.073405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:02:58.073405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 112/238] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:58.079497 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35be667-917e-4c85-83fa-7c54b61c887c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:58.083898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b2a5ead5-2e7e-4677-9968-b93f92d9c8e0 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:58.085412 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b2a5ead5-2e7e-4677-9968-b93f92d9c8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:58.163995 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:58.167084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:58.207277 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 127/239] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:58.247026 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35be667-917e-4c85-83fa-7c54b61c887c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:58.269408 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5ead5-2e7e-4677-9968-b93f92d9c8e0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:58.296011 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9e677-ad63-41ea-bb70-74ccbfacde07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:58.296701 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9e677-ad63-41ea-bb70-74ccbfacde07 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:58.296983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9e677-ad63-41ea-bb70-74ccbfacde07 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:58.297355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9e677-ad63-41ea-bb70-74ccbfacde07 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:58.298120 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9e677-ad63-41ea-bb70-74ccbfacde07 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:58.298426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9e677-ad63-41ea-bb70-74ccbfacde07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:58.299955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9e677-ad63-41ea-bb70-74ccbfacde07 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:58.309743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 113/240] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:58.315270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9e677-ad63-41ea-bb70-74ccbfacde07 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:58.383512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5ead5-2e7e-4677-9968-b93f92d9c8e0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:58.422678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbce950-0a6f-43c8-a30e-9ffd4662fe32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:58.423485 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbce950-0a6f-43c8-a30e-9ffd4662fe32 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:58.423804 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbce950-0a6f-43c8-a30e-9ffd4662fe32 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:58.424097 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbce950-0a6f-43c8-a30e-9ffd4662fe32 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:58.425284 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbce950-0a6f-43c8-a30e-9ffd4662fe32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:58.426756 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbce950-0a6f-43c8-a30e-9ffd4662fe32 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:58.428762 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 128/241] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:58.434500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbce950-0a6f-43c8-a30e-9ffd4662fe32 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:58.440252 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9e677-ad63-41ea-bb70-74ccbfacde07 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:58.484991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:58.494424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:58.540335 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1cb22d-4ee3-445f-8074-fccd6be58ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:58.541254 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cb22d-4ee3-445f-8074-fccd6be58ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:58.541541 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cb22d-4ee3-445f-8074-fccd6be58ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:58.541819 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cb22d-4ee3-445f-8074-fccd6be58ea0 None admin] PATH_INFO: `/v3/users/437efdcce5b94e21a41c9357b64beba5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:58.554812 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbce950-0a6f-43c8-a30e-9ffd4662fe32 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:58.574058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1cb22d-4ee3-445f-8074-fccd6be58ea0 None admin] RBAC: Authorizing `identity:get_user(user_id=437efdcce5b94e21a41c9357b64beba5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:58.575253 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1cb22d-4ee3-445f-8074-fccd6be58ea0 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:58.578005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 114/242] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:58 2022] GET /identity/v3/users/437efdcce5b94e21a41c9357b64beba5 => generated 297 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:58.588423 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1cb22d-4ee3-445f-8074-fccd6be58ea0 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:58.601494 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01db63-6f46-4c7f-a15d-562f9805684c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:58.602487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01db63-6f46-4c7f-a15d-562f9805684c None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:58.602957 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01db63-6f46-4c7f-a15d-562f9805684c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:58.603460 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01db63-6f46-4c7f-a15d-562f9805684c None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:58.604383 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01db63-6f46-4c7f-a15d-562f9805684c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:58.606049 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01db63-6f46-4c7f-a15d-562f9805684c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:58.631948 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 129/243] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Jun 27 21:02:58 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:02:58.724151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfd6b0-2db0-4bfc-81fd-d45076e9c55d None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:58.724804 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfd6b0-2db0-4bfc-81fd-d45076e9c55d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:58.725270 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfd6b0-2db0-4bfc-81fd-d45076e9c55d None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:58.726835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 130/244] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:02:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:58.732367 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafab6c-646e-41df-a468-9458846808c8 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:58.732736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafab6c-646e-41df-a468-9458846808c8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:58.732927 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafab6c-646e-41df-a468-9458846808c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:58.768124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-baafab6c-646e-41df-a468-9458846808c8 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:58.770312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-baafab6c-646e-41df-a468-9458846808c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:58.772673 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1cb22d-4ee3-445f-8074-fccd6be58ea0 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:58.840515 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eec636-ad1f-4dab-a2fb-440851b3dc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:58.841105 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eec636-ad1f-4dab-a2fb-440851b3dc51 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:58.841388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eec636-ad1f-4dab-a2fb-440851b3dc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:58.841665 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eec636-ad1f-4dab-a2fb-440851b3dc51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:58.884660 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 131/245] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:58.904585 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:58.907307 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafab6c-646e-41df-a468-9458846808c8 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:58.922758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eec636-ad1f-4dab-a2fb-440851b3dc51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:58.924704 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eec636-ad1f-4dab-a2fb-440851b3dc51 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:58.945797 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:59.023687 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafab6c-646e-41df-a468-9458846808c8 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:59.034324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 115/246] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 21:02:59.041281 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eec636-ad1f-4dab-a2fb-440851b3dc51 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:59.069911 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a897df4-c955-4bdd-8ed7-8dafbd49124f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:59.071528 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a897df4-c955-4bdd-8ed7-8dafbd49124f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:59.072166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a897df4-c955-4bdd-8ed7-8dafbd49124f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:59.072708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a897df4-c955-4bdd-8ed7-8dafbd49124f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:59.085600 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a897df4-c955-4bdd-8ed7-8dafbd49124f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:59.095593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-3a897df4-c955-4bdd-8ed7-8dafbd49124f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:59.095593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:59.095593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:59.095593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:59.095593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:59.095593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:59.095593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:59.095593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:59.095593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:59.095593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:59.095593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:59.095593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:59.095593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:59.095593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:59.095593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:59.095593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:59.096790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:59.096790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:59.096790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:59.096790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:59.096790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:59.096790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:59.096790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:59.096790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:59.096790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:59.096790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:59.096790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:59.096790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:59.096790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:59.096790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:59.096790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:59.096790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:59.097757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:59.097757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:59.097757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:59.097757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:59.097757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:59.097757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:59.097757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:59.097757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:59.097757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:59.097757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:59.097757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:59.097757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:02:59.100488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 132/247] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:59.105959 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a897df4-c955-4bdd-8ed7-8dafbd49124f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:59.134116 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:59.170268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eec636-ad1f-4dab-a2fb-440851b3dc51 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:59.210910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb62122-addc-4233-ad1d-c888b802b81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:59.212021 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb62122-addc-4233-ad1d-c888b802b81a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:59.212405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb62122-addc-4233-ad1d-c888b802b81a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:59.212867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb62122-addc-4233-ad1d-c888b802b81a None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:59.214085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb62122-addc-4233-ad1d-c888b802b81a None admin] RBAC: Authorizing `identity:get_project(project_id=e7099ea37d874a7aa73b15453887dffc)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:59.216161 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb62122-addc-4233-ad1d-c888b802b81a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:59.218350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 116/248] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:59 2022] GET /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:59.224047 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:59.224388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb62122-addc-4233-ad1d-c888b802b81a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:59.239855 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a897df4-c955-4bdd-8ed7-8dafbd49124f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:59.292361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5643e-426d-4a7b-bb7f-19e9b46221d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:59.293182 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5643e-426d-4a7b-bb7f-19e9b46221d2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:59.293461 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5643e-426d-4a7b-bb7f-19e9b46221d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:59.293733 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5643e-426d-4a7b-bb7f-19e9b46221d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:59.294487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5643e-426d-4a7b-bb7f-19e9b46221d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:59.294894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5643e-426d-4a7b-bb7f-19e9b46221d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:59.296450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5643e-426d-4a7b-bb7f-19e9b46221d2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:59.306935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 133/249] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:59.311939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5643e-426d-4a7b-bb7f-19e9b46221d2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:59.341084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb62122-addc-4233-ad1d-c888b802b81a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:59.381963 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e7627-878f-4b0b-a8fe-53370374f297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:59.382854 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e7627-878f-4b0b-a8fe-53370374f297 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:59.383279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e7627-878f-4b0b-a8fe-53370374f297 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:59.383766 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e7627-878f-4b0b-a8fe-53370374f297 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:59.385251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9e7627-878f-4b0b-a8fe-53370374f297 None admin] RBAC: Adding query filter params (role.id=e71d790e22744442b6ff4af3caa1e853, scope.project.id=e7099ea37d874a7aa73b15453887dffc, user.id=437efdcce5b94e21a41c9357b64beba5) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:59.385754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9e7627-878f-4b0b-a8fe-53370374f297 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:59.387449 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9e7627-878f-4b0b-a8fe-53370374f297 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:59.394573 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 117/250] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:59 2022] GET /identity/v3/role_assignments?user.id=437efdcce5b94e21a41c9357b64beba5&scope.project.id=e7099ea37d874a7aa73b15453887dffc&role.id=e71d790e22744442b6ff4af3caa1e853 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:59.495764 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5643e-426d-4a7b-bb7f-19e9b46221d2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:59.538031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b050dc-36cb-492d-aaf5-e017426a852c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:59.538938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b050dc-36cb-492d-aaf5-e017426a852c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:59.539215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b050dc-36cb-492d-aaf5-e017426a852c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:59.539488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b050dc-36cb-492d-aaf5-e017426a852c None admin] PATH_INFO: `/v3/users/5864827f36c54aa89f4bc422ba9d8874` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:59.540740 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b050dc-36cb-492d-aaf5-e017426a852c None admin] RBAC: Authorizing `identity:get_user(user_id=5864827f36c54aa89f4bc422ba9d8874)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:59.542222 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b050dc-36cb-492d-aaf5-e017426a852c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:59.543921 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 134/251] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:59 2022] GET /identity/v3/users/5864827f36c54aa89f4bc422ba9d8874 => generated 297 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:59.549794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e7627-878f-4b0b-a8fe-53370374f297 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:59.570209 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:59.692457 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e7627-878f-4b0b-a8fe-53370374f297 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:59.737260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680293e3-359e-470d-a0da-bc7e13eebefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:59.738159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680293e3-359e-470d-a0da-bc7e13eebefd None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:59.738439 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680293e3-359e-470d-a0da-bc7e13eebefd None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:59.738714 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680293e3-359e-470d-a0da-bc7e13eebefd None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:59.739958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680293e3-359e-470d-a0da-bc7e13eebefd None admin] RBAC: Authorizing `identity:get_project(project_id=e7099ea37d874a7aa73b15453887dffc)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:59.741441 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680293e3-359e-470d-a0da-bc7e13eebefd None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:59.743174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 118/252] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:59 2022] GET /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:59.749034 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b050dc-36cb-492d-aaf5-e017426a852c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:59.794431 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:59.884004 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b050dc-36cb-492d-aaf5-e017426a852c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:59.934614 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbca13-1f11-4a5f-9591-d6b2be7f026a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:59.935529 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbca13-1f11-4a5f-9591-d6b2be7f026a None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:59.935874 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbca13-1f11-4a5f-9591-d6b2be7f026a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:59.936150 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbca13-1f11-4a5f-9591-d6b2be7f026a None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc/users/5864827f36c54aa89f4bc422ba9d8874/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:59.940938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfbca13-1f11-4a5f-9591-d6b2be7f026a None admin] RBAC: Authorizing `identity:create_grant(project_id=e7099ea37d874a7aa73b15453887dffc, user_id=5864827f36c54aa89f4bc422ba9d8874, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:59.942421 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfbca13-1f11-4a5f-9591-d6b2be7f026a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:59.963500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 135/253] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:02:59 2022] PUT /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc/users/5864827f36c54aa89f4bc422ba9d8874/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:00.581872 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6142fb-8dd2-485c-81eb-81dfa921ae71 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:00.582419 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6142fb-8dd2-485c-81eb-81dfa921ae71 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:00.582419 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6142fb-8dd2-485c-81eb-81dfa921ae71 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:00.584087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 119/254] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:00.589413 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea713f4-b5c2-41ed-a44d-4986c0b2eda5 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:00.589703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea713f4-b5c2-41ed-a44d-4986c0b2eda5 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:00.589980 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea713f4-b5c2-41ed-a44d-4986c0b2eda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:00.616608 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7ea713f4-b5c2-41ed-a44d-4986c0b2eda5 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:00.617809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7ea713f4-b5c2-41ed-a44d-4986c0b2eda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:00.700035 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 136/255] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:00.723716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6142fb-8dd2-485c-81eb-81dfa921ae71 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:00.804681 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:00.835577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6142fb-8dd2-485c-81eb-81dfa921ae71 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:00.880068 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1abad5-5a12-46ca-b03c-9aef9ecc157b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:00.881273 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1abad5-5a12-46ca-b03c-9aef9ecc157b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:00.881764 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1abad5-5a12-46ca-b03c-9aef9ecc157b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:00.882224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1abad5-5a12-46ca-b03c-9aef9ecc157b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:00.893276 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1abad5-5a12-46ca-b03c-9aef9ecc157b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:00.901421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-dd1abad5-5a12-46ca-b03c-9aef9ecc157b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:00.901421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:00.901421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:00.901421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:00.901421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:00.901421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:00.901421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:00.901421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:00.901421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:00.901421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:00.901421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:00.901421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:00.901421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:00.901421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:00.901421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:00.901421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:00.902278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:00.902278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:00.902278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:00.902278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:00.902278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:00.902278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:00.902278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:00.902278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:00.902278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:00.902278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:00.902278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:00.902278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:00.902278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:00.902278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:00.902278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:00.902278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:00.903058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:00.903058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:00.903058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:00.903058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:00.903058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:00.903058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:00.903058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:00.903058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:00.903058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:00.903058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:00.903058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:00.903058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:00.905718 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 120/256] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:00.912170 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea713f4-b5c2-41ed-a44d-4986c0b2eda5 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:00.928327 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:01.045312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea713f4-b5c2-41ed-a44d-4986c0b2eda5 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:01.088923 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57c22a-da85-4e22-91bd-2eab140bae65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:01.089943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57c22a-da85-4e22-91bd-2eab140bae65 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:01.090410 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57c22a-da85-4e22-91bd-2eab140bae65 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:01.090883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57c22a-da85-4e22-91bd-2eab140bae65 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:01.091846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57c22a-da85-4e22-91bd-2eab140bae65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:01.092354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57c22a-da85-4e22-91bd-2eab140bae65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:01.093944 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57c22a-da85-4e22-91bd-2eab140bae65 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:01.102895 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 137/257] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:01.108591 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1abad5-5a12-46ca-b03c-9aef9ecc157b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:01.195663 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e00d0-b62a-463d-bf0c-da4999b347be None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:01.196298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e00d0-b62a-463d-bf0c-da4999b347be None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:01.196829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e00d0-b62a-463d-bf0c-da4999b347be None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:01.198382 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 138/258] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:01.205006 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c66ca9-2b03-4d32-8d53-08fbb4672a29 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:01.205527 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c66ca9-2b03-4d32-8d53-08fbb4672a29 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:01.206014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c66ca9-2b03-4d32-8d53-08fbb4672a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:01.216610 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1abad5-5a12-46ca-b03c-9aef9ecc157b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:01.231740 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f9c66ca9-2b03-4d32-8d53-08fbb4672a29 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:01.233210 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f9c66ca9-2b03-4d32-8d53-08fbb4672a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:01.300478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0a15e-8ee2-416c-a902-9ee6dd1833a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:01.307629 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0a15e-8ee2-416c-a902-9ee6dd1833a6 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:01.310080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0a15e-8ee2-416c-a902-9ee6dd1833a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:01.310569 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0a15e-8ee2-416c-a902-9ee6dd1833a6 None admin] PATH_INFO: `/v3/users/f5bb1f06e8d942729c697de95e5b0448` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:01.312239 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0a15e-8ee2-416c-a902-9ee6dd1833a6 None admin] RBAC: Authorizing `identity:get_user(user_id=f5bb1f06e8d942729c697de95e5b0448)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:01.314584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0a15e-8ee2-416c-a902-9ee6dd1833a6 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:01.316696 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 121/259] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:03:01 2022] GET /identity/v3/users/f5bb1f06e8d942729c697de95e5b0448 => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:01.330287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0a15e-8ee2-416c-a902-9ee6dd1833a6 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:01.338809 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:01.406368 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 139/260] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:01.472955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c66ca9-2b03-4d32-8d53-08fbb4672a29 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:01.486244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:01.501954 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0a15e-8ee2-416c-a902-9ee6dd1833a6 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:01.540154 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f5b11-d4ec-47d1-bbec-3ca4173a4dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:01.540827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f5b11-d4ec-47d1-bbec-3ca4173a4dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:01.541366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f5b11-d4ec-47d1-bbec-3ca4173a4dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:01.541652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f5b11-d4ec-47d1-bbec-3ca4173a4dd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:01.663153 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c66ca9-2b03-4d32-8d53-08fbb4672a29 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:01.668471 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f5b11-d4ec-47d1-bbec-3ca4173a4dd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:01.669993 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f5b11-d4ec-47d1-bbec-3ca4173a4dd9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:01.672961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:01.704707 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cec005-a540-4adf-bb1b-60c00de6e71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:01.705602 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cec005-a540-4adf-bb1b-60c00de6e71b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:01.705885 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cec005-a540-4adf-bb1b-60c00de6e71b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:01.706160 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cec005-a540-4adf-bb1b-60c00de6e71b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:01.717385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cec005-a540-4adf-bb1b-60c00de6e71b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:01.729116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-75cec005-a540-4adf-bb1b-60c00de6e71b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:03:01.729116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:01.729116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:01.729116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:01.729116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:01.729116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:01.729116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:01.729116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:01.729116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:01.729116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:01.729116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:01.729116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:01.729116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:01.729116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:01.729116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:01.729116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:01.730155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:01.730155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:01.730155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:01.730155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:01.730155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:01.730155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:01.730155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:01.730155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:01.730155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:01.730155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:01.730155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:01.730155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:01.730155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:01.730155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:01.730155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:01.730155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:01.730952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:01.730952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:01.730952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:01.730952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:01.730952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:01.730952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:01.730952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:01.730952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:01.730952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:01.730952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:01.730952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:03:01.730952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:01.732947 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 140/261] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:03:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:01.738157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cec005-a540-4adf-bb1b-60c00de6e71b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:01.806988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:01.845832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cec005-a540-4adf-bb1b-60c00de6e71b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:01.881104 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da004580-4028-4e09-8286-bb11c6f7bd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:01.881792 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da004580-4028-4e09-8286-bb11c6f7bd0b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:01.882260 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da004580-4028-4e09-8286-bb11c6f7bd0b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:01.882538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da004580-4028-4e09-8286-bb11c6f7bd0b None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:01.883286 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da004580-4028-4e09-8286-bb11c6f7bd0b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:01.883597 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da004580-4028-4e09-8286-bb11c6f7bd0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:01.885010 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da004580-4028-4e09-8286-bb11c6f7bd0b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:01.893859 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 141/262] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:01.895111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111dd44-391c-4a7b-a5c6-6e742f7c5e52 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:01.895399 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111dd44-391c-4a7b-a5c6-6e742f7c5e52 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:01.899802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111dd44-391c-4a7b-a5c6-6e742f7c5e52 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:01.901341 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 142/263] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:01.902588 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111dd44-391c-4a7b-a5c6-6e742f7c5e52 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:01.916729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 122/264] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:01.918307 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efde8bd-6208-4b82-b9b4-0c5f50fbf0b7 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:01.918796 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efde8bd-6208-4b82-b9b4-0c5f50fbf0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:01.919268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efde8bd-6208-4b82-b9b4-0c5f50fbf0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:01.952596 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-0efde8bd-6208-4b82-b9b4-0c5f50fbf0b7 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:01.954219 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0efde8bd-6208-4b82-b9b4-0c5f50fbf0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:02.008994 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111dd44-391c-4a7b-a5c6-6e742f7c5e52 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:02.037167 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:02.046914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a97ee-d685-47e0-9381-2f7a8b15d6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:02.048127 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a97ee-d685-47e0-9381-2f7a8b15d6ab None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:02.048629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a97ee-d685-47e0-9381-2f7a8b15d6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:02.049130 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a97ee-d685-47e0-9381-2f7a8b15d6ab None admin] PATH_INFO: `/v3/users/f5bb1f06e8d942729c697de95e5b0448` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:02.050560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a97ee-d685-47e0-9381-2f7a8b15d6ab None admin] RBAC: Authorizing `identity:get_user(user_id=f5bb1f06e8d942729c697de95e5b0448)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:02.052286 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a97ee-d685-47e0-9381-2f7a8b15d6ab None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:02.054433 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 143/265] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:03:01 2022] GET /identity/v3/users/f5bb1f06e8d942729c697de95e5b0448 => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:02.055400 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a97ee-d685-47e0-9381-2f7a8b15d6ab None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:02.102341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 123/266] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:02.103207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efde8bd-6208-4b82-b9b4-0c5f50fbf0b7 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:02.166125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:02.166448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a97ee-d685-47e0-9381-2f7a8b15d6ab None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:02.202756 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7286ee4-2ecb-40d5-817b-f8d8bd63195c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:02.204009 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7286ee4-2ecb-40d5-817b-f8d8bd63195c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:02.204505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7286ee4-2ecb-40d5-817b-f8d8bd63195c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:02.204970 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7286ee4-2ecb-40d5-817b-f8d8bd63195c None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:02.206408 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7286ee4-2ecb-40d5-817b-f8d8bd63195c None admin] RBAC: Authorizing `identity:get_project(project_id=e7099ea37d874a7aa73b15453887dffc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:02.208108 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7286ee4-2ecb-40d5-817b-f8d8bd63195c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:02.210132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 144/267] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:03:02 2022] GET /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:02.211077 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7286ee4-2ecb-40d5-817b-f8d8bd63195c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:02.238492 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efde8bd-6208-4b82-b9b4-0c5f50fbf0b7 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:02.274547 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140cfab-9217-4779-8386-81c0c6ebb728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:02.275227 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140cfab-9217-4779-8386-81c0c6ebb728 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:02.275517 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140cfab-9217-4779-8386-81c0c6ebb728 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:02.275904 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140cfab-9217-4779-8386-81c0c6ebb728 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:02.276997 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:02.319472 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7286ee4-2ecb-40d5-817b-f8d8bd63195c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:02.349053 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140cfab-9217-4779-8386-81c0c6ebb728 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:02.350566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140cfab-9217-4779-8386-81c0c6ebb728 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:02.357167 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0683cf1-b642-4a59-985f-96a3f8de72fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:02.358228 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0683cf1-b642-4a59-985f-96a3f8de72fb None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:02.358721 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0683cf1-b642-4a59-985f-96a3f8de72fb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:02.359187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0683cf1-b642-4a59-985f-96a3f8de72fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:02.370560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0683cf1-b642-4a59-985f-96a3f8de72fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:02.378866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f0683cf1-b642-4a59-985f-96a3f8de72fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:03:02.378866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:02.378866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:02.378866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:02.378866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:02.378866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:02.378866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:02.378866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:02.378866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:02.378866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:02.378866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:02.378866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:02.378866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:02.378866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:02.378866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:02.378866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:02.379789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:02.379789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:02.379789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:02.379789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:02.379789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:02.379789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:02.379789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:02.379789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:02.379789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:02.379789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:02.379789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:02.379789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:02.379789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:02.379789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:02.379789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:02.379789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:02.380574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:02.380574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:02.380574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:02.380574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:02.380574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:02.380574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:02.380574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:02.380574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:02.380574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:02.380574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:02.380574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:03:02.380574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:02.382949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 145/268] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:02.384126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0683cf1-b642-4a59-985f-96a3f8de72fb None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:02.390258 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:02.462401 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 124/269] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:02.463015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140cfab-9217-4779-8386-81c0c6ebb728 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:02.496679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0683cf1-b642-4a59-985f-96a3f8de72fb None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:02.521616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:02.535113 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec52e8-8a48-4c13-b7cf-3426a1ba7558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:02.536158 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec52e8-8a48-4c13-b7cf-3426a1ba7558 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:02.536666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec52e8-8a48-4c13-b7cf-3426a1ba7558 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:02.537132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec52e8-8a48-4c13-b7cf-3426a1ba7558 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:02.538160 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec52e8-8a48-4c13-b7cf-3426a1ba7558 None admin] RBAC: Adding query filter params (role.id=5c3c095bb53943eca95c4e049cd8cbf5, scope.project.id=e7099ea37d874a7aa73b15453887dffc, user.id=f5bb1f06e8d942729c697de95e5b0448) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:02.538666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec52e8-8a48-4c13-b7cf-3426a1ba7558 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:02.540360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec52e8-8a48-4c13-b7cf-3426a1ba7558 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:02.546871 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 146/270] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:02 2022] GET /identity/v3/role_assignments?user.id=f5bb1f06e8d942729c697de95e5b0448&scope.project.id=e7099ea37d874a7aa73b15453887dffc&role.id=5c3c095bb53943eca95c4e049cd8cbf5 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:02.547964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec52e8-8a48-4c13-b7cf-3426a1ba7558 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:02.570289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140cfab-9217-4779-8386-81c0c6ebb728 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:02.608199 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b01ce-32a2-42f6-b145-0547a482450c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:02.608895 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b01ce-32a2-42f6-b145-0547a482450c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:02.609180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b01ce-32a2-42f6-b145-0547a482450c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:02.609458 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b01ce-32a2-42f6-b145-0547a482450c None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:02.610228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b01ce-32a2-42f6-b145-0547a482450c None admin] RBAC: Adding query filter params (name=member) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:02.610539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b01ce-32a2-42f6-b145-0547a482450c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:02.612347 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b01ce-32a2-42f6-b145-0547a482450c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:02.613682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:02.622493 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 125/271] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:03:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:02.628501 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ed142-4ef9-4669-b220-0da2ff2335b2 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:02.628860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ed142-4ef9-4669-b220-0da2ff2335b2 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:02.629155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ed142-4ef9-4669-b220-0da2ff2335b2 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:02.630713 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 126/272] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:02.632113 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdf6c9-48ea-448e-a11f-cf69e5f7a6b5 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:02.632400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdf6c9-48ea-448e-a11f-cf69e5f7a6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:02.632691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdf6c9-48ea-448e-a11f-cf69e5f7a6b5 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:02.634001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 127/273] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:02.634706 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffdf6c9-48ea-448e-a11f-cf69e5f7a6b5 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:02.669233 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec52e8-8a48-4c13-b7cf-3426a1ba7558 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:02.710456 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391aae40-75f0-4b23-a4a5-d7ed28856c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:02.711774 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391aae40-75f0-4b23-a4a5-d7ed28856c5d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:02.712260 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391aae40-75f0-4b23-a4a5-d7ed28856c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:02.712910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391aae40-75f0-4b23-a4a5-d7ed28856c5d None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:02.714512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391aae40-75f0-4b23-a4a5-d7ed28856c5d None admin] RBAC: Authorizing `identity:get_project(project_id=e7099ea37d874a7aa73b15453887dffc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:02.716261 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391aae40-75f0-4b23-a4a5-d7ed28856c5d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:02.718271 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 147/274] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:03:02 2022] GET /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:02.719888 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b94269-5ac0-4405-a50e-c720ac97e596 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:02.720481 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b94269-5ac0-4405-a50e-c720ac97e596 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:02.721020 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b94269-5ac0-4405-a50e-c720ac97e596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:02.739444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-26b94269-5ac0-4405-a50e-c720ac97e596 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:02.741181 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-26b94269-5ac0-4405-a50e-c720ac97e596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:02.747964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffdf6c9-48ea-448e-a11f-cf69e5f7a6b5 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:02.784905 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a119be-41dc-44fb-9874-66fa0c57343c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:02.785987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a119be-41dc-44fb-9874-66fa0c57343c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:02.786363 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a119be-41dc-44fb-9874-66fa0c57343c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:02.786645 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a119be-41dc-44fb-9874-66fa0c57343c None admin] PATH_INFO: `/v3/users/2fa15e34127043dbb4646ac5a8a32903` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:02.801394 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a119be-41dc-44fb-9874-66fa0c57343c None admin] RBAC: Authorizing `identity:get_user(user_id=2fa15e34127043dbb4646ac5a8a32903)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:02.802893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a119be-41dc-44fb-9874-66fa0c57343c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:02.805637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 128/275] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:03:02 2022] GET /identity/v3/users/2fa15e34127043dbb4646ac5a8a32903 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:02.806584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a64113-1e60-41fd-a29c-67de9c87e897 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:02.806872 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a64113-1e60-41fd-a29c-67de9c87e897 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:02.807153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a64113-1e60-41fd-a29c-67de9c87e897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:02.807364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:02.820260 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 148/276] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:02.821420 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b94269-5ac0-4405-a50e-c720ac97e596 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:02.829797 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-17a64113-1e60-41fd-a29c-67de9c87e897 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:02.831102 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-17a64113-1e60-41fd-a29c-67de9c87e897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:02.886393 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:02.905667 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 129/277] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:02.906510 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a64113-1e60-41fd-a29c-67de9c87e897 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:02.932567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b94269-5ac0-4405-a50e-c720ac97e596 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:02.966007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012955b8-851a-4947-8d35-1374c1edd6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:02.966698 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012955b8-851a-4947-8d35-1374c1edd6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:02.966984 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012955b8-851a-4947-8d35-1374c1edd6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:02.967279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012955b8-851a-4947-8d35-1374c1edd6c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:02.968219 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012955b8-851a-4947-8d35-1374c1edd6c0 None admin] RBAC: Adding query filter params (role.id=b3efb2b2e43c4af8b1cf36a37f1e05f9, scope.project.id=e7099ea37d874a7aa73b15453887dffc, user.id=f5bb1f06e8d942729c697de95e5b0448) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:02.968534 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012955b8-851a-4947-8d35-1374c1edd6c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:02.970057 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012955b8-851a-4947-8d35-1374c1edd6c0 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:02.991638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 149/278] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:02 2022] GET /identity/v3/role_assignments?user.id=f5bb1f06e8d942729c697de95e5b0448&scope.project.id=e7099ea37d874a7aa73b15453887dffc&role.id=b3efb2b2e43c4af8b1cf36a37f1e05f9 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:02.991638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012955b8-851a-4947-8d35-1374c1edd6c0 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:03.014318 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a64113-1e60-41fd-a29c-67de9c87e897 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:03.037226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:03.052588 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719f713-41b2-4059-8451-268975f2d493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:03.053396 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719f713-41b2-4059-8451-268975f2d493 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.054000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719f713-41b2-4059-8451-268975f2d493 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.054000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719f713-41b2-4059-8451-268975f2d493 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.054967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719f713-41b2-4059-8451-268975f2d493 None admin] RBAC: Adding query filter params (role.id=b1bcb7445ee54636ae7b29a2485cc201, user.id=2fa15e34127043dbb4646ac5a8a32903, scope.system=all) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:03.055345 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719f713-41b2-4059-8451-268975f2d493 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:03.056935 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719f713-41b2-4059-8451-268975f2d493 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:03.063377 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 130/279] 10.222.0.27 () {56 vars in 1308 bytes} [Mon Jun 27 21:03:02 2022] GET /identity/v3/role_assignments?user.id=2fa15e34127043dbb4646ac5a8a32903&scope.system=all&role.id=b1bcb7445ee54636ae7b29a2485cc201 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:03.064930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675e5a5-973f-4ed2-8e53-448d513ee814 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.065311 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675e5a5-973f-4ed2-8e53-448d513ee814 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.065660 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675e5a5-973f-4ed2-8e53-448d513ee814 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.067337 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 131/280] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:03.068061 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675e5a5-973f-4ed2-8e53-448d513ee814 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:03.124490 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012955b8-851a-4947-8d35-1374c1edd6c0 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:03.130257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:03.177696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974dc31-eb7c-4194-bdbe-bce1e3bc9418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:03.178831 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974dc31-eb7c-4194-bdbe-bce1e3bc9418 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.179253 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974dc31-eb7c-4194-bdbe-bce1e3bc9418 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.179789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974dc31-eb7c-4194-bdbe-bce1e3bc9418 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.182078 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974dc31-eb7c-4194-bdbe-bce1e3bc9418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:03.183870 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974dc31-eb7c-4194-bdbe-bce1e3bc9418 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:03.185832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 150/281] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:03.187299 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d779209-988e-4564-9507-cff71fc72f11 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.187757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d779209-988e-4564-9507-cff71fc72f11 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.188037 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d779209-988e-4564-9507-cff71fc72f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.199600 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675e5a5-973f-4ed2-8e53-448d513ee814 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:03.206126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4d779209-988e-4564-9507-cff71fc72f11 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:03.207362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4d779209-988e-4564-9507-cff71fc72f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:03.244690 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7b35a-cd58-42d4-ad77-3fe00e02e819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:03.245829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7b35a-cd58-42d4-ad77-3fe00e02e819 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.246326 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7b35a-cd58-42d4-ad77-3fe00e02e819 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.246792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7b35a-cd58-42d4-ad77-3fe00e02e819 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.259844 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7b35a-cd58-42d4-ad77-3fe00e02e819 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:03.268940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-a9d7b35a-cd58-42d4-ad77-3fe00e02e819 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:03:03.268940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:03.268940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:03.268940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:03.268940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:03.268940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:03.268940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:03.268940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:03.268940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:03.268940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:03.268940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:03.268940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:03.268940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:03.268940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:03.268940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:03.268940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:03.269851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:03.269851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:03.269851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:03.269851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:03.269851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:03.269851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:03.269851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:03.269851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:03.269851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:03.269851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:03.269851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:03.269851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:03.269851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:03.269851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:03.269851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:03.269851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:03.270623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:03.270623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:03.270623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:03.270623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:03.270623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:03.270623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:03.270623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:03.270623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:03.270623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:03.270623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:03.270623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:03:03.270623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:03.273453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 132/282] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:03.274260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7b35a-cd58-42d4-ad77-3fe00e02e819 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:03.294787 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 151/283] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:03.295982 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d779209-988e-4564-9507-cff71fc72f11 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:03.345544 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:03.397057 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7b35a-cd58-42d4-ad77-3fe00e02e819 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:03.398988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d779209-988e-4564-9507-cff71fc72f11 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:03.408036 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:03.435261 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87dad4-5fa7-4b1f-b043-6774461ddb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:03.435968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87dad4-5fa7-4b1f-b043-6774461ddb5f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.436250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87dad4-5fa7-4b1f-b043-6774461ddb5f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.436368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a4027-e7c3-4d8e-a517-cd0cd8a348fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:03.436556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87dad4-5fa7-4b1f-b043-6774461ddb5f None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.437008 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a4027-e7c3-4d8e-a517-cd0cd8a348fa None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.437293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a4027-e7c3-4d8e-a517-cd0cd8a348fa None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.437382 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87dad4-5fa7-4b1f-b043-6774461ddb5f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:03.437570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a4027-e7c3-4d8e-a517-cd0cd8a348fa None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.437660 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87dad4-5fa7-4b1f-b043-6774461ddb5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:03.438287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a4027-e7c3-4d8e-a517-cd0cd8a348fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:03.439000 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87dad4-5fa7-4b1f-b043-6774461ddb5f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:03.439858 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a4027-e7c3-4d8e-a517-cd0cd8a348fa None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:03.452665 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 152/284] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:03:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:03.452665 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87dad4-5fa7-4b1f-b043-6774461ddb5f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:03.464596 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 133/285] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Jun 27 21:03:03 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:03.466142 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a4027-e7c3-4d8e-a517-cd0cd8a348fa None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:03.567314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87dad4-5fa7-4b1f-b043-6774461ddb5f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:03.588004 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:03.610595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a4027-e7c3-4d8e-a517-cd0cd8a348fa None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:03.614445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db98dea-b6fd-4b0b-ad05-0cddcb93af20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:03.615445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db98dea-b6fd-4b0b-ad05-0cddcb93af20 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.615885 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db98dea-b6fd-4b0b-ad05-0cddcb93af20 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.616256 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db98dea-b6fd-4b0b-ad05-0cddcb93af20 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.631066 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db98dea-b6fd-4b0b-ad05-0cddcb93af20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:03.641288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4db98dea-b6fd-4b0b-ad05-0cddcb93af20 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:03:03.641288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:03.641288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:03.641288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:03.641288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:03.641288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:03.641288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:03.641288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:03.641288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:03.641288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:03.641288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:03.641288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:03.641288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:03.641288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:03.641288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:03.641288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:03.642183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:03.642183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:03.642183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:03.642183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:03.642183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:03.642183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:03.642183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:03.642183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:03.642183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:03.642183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:03.642183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:03.642183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:03.642183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:03.642183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:03.642183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:03.642183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:03.643587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:03.643587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:03.643587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:03.643587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:03.643587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:03.643587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:03.643587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:03.643587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:03.643587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:03.643587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:03.643587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:03:03.643587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:03.643587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 153/286] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:03.648816 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db98dea-b6fd-4b0b-ad05-0cddcb93af20 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:03.654687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b16886-55f1-4788-9883-31485345749b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:03.655594 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b16886-55f1-4788-9883-31485345749b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.655904 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b16886-55f1-4788-9883-31485345749b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.656181 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b16886-55f1-4788-9883-31485345749b None admin] PATH_INFO: `/v3/users/437efdcce5b94e21a41c9357b64beba5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.657394 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b16886-55f1-4788-9883-31485345749b None admin] RBAC: Authorizing `identity:get_user(user_id=437efdcce5b94e21a41c9357b64beba5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:03.659229 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b16886-55f1-4788-9883-31485345749b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:03.661011 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 134/287] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:03:03 2022] GET /identity/v3/users/437efdcce5b94e21a41c9357b64beba5 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:03.666428 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b16886-55f1-4788-9883-31485345749b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:03.713912 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:03.793411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db98dea-b6fd-4b0b-ad05-0cddcb93af20 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:03.807065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b16886-55f1-4788-9883-31485345749b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:03.841401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249691ee-4051-4d4a-902b-b10b452e3430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:03.842010 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249691ee-4051-4d4a-902b-b10b452e3430 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.842291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249691ee-4051-4d4a-902b-b10b452e3430 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.842565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249691ee-4051-4d4a-902b-b10b452e3430 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.843366 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249691ee-4051-4d4a-902b-b10b452e3430 None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:03.843647 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249691ee-4051-4d4a-902b-b10b452e3430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:03.846135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249691ee-4051-4d4a-902b-b10b452e3430 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:03.846320 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c252bf9-0130-4827-ada0-d6fedc6bc1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:03.847213 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c252bf9-0130-4827-ada0-d6fedc6bc1ea None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.847488 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c252bf9-0130-4827-ada0-d6fedc6bc1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.847991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c252bf9-0130-4827-ada0-d6fedc6bc1ea None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.849627 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c252bf9-0130-4827-ada0-d6fedc6bc1ea None admin] RBAC: Authorizing `identity:get_project(project_id=e7099ea37d874a7aa73b15453887dffc)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:03.851111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c252bf9-0130-4827-ada0-d6fedc6bc1ea None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:03.853752 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 135/288] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:03:03 2022] GET /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:03.858985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 154/289] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:03:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:03.859921 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c252bf9-0130-4827-ada0-d6fedc6bc1ea None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:03.865394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249691ee-4051-4d4a-902b-b10b452e3430 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:03.907423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:03.987281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249691ee-4051-4d4a-902b-b10b452e3430 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:04.000809 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:04.007635 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c252bf9-0130-4827-ada0-d6fedc6bc1ea None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:04.049289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8948a-d4e7-4093-a8db-1f2d86813979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:04.050186 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8948a-d4e7-4093-a8db-1f2d86813979 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:04.050463 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8948a-d4e7-4093-a8db-1f2d86813979 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:04.050735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8948a-d4e7-4093-a8db-1f2d86813979 None admin] PATH_INFO: `/v3/users/5864827f36c54aa89f4bc422ba9d8874` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:04.052588 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a8948a-d4e7-4093-a8db-1f2d86813979 None admin] RBAC: Authorizing `identity:get_user(user_id=5864827f36c54aa89f4bc422ba9d8874)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:04.054053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a8948a-d4e7-4093-a8db-1f2d86813979 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:04.055910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 155/290] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:03:03 2022] GET /identity/v3/users/5864827f36c54aa89f4bc422ba9d8874 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:04.061713 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8948a-d4e7-4093-a8db-1f2d86813979 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:04.075744 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64d8c5-f92b-4081-b4d3-6d8d09a97fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:04.076658 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64d8c5-f92b-4081-b4d3-6d8d09a97fdd None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:04.076940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64d8c5-f92b-4081-b4d3-6d8d09a97fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:04.077216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64d8c5-f92b-4081-b4d3-6d8d09a97fdd None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc/users/437efdcce5b94e21a41c9357b64beba5/roles/e71d790e22744442b6ff4af3caa1e853` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:04.082666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64d8c5-f92b-4081-b4d3-6d8d09a97fdd None admin] RBAC: Authorizing `identity:create_grant(project_id=e7099ea37d874a7aa73b15453887dffc, user_id=437efdcce5b94e21a41c9357b64beba5, role_id=e71d790e22744442b6ff4af3caa1e853)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:04.084161 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64d8c5-f92b-4081-b4d3-6d8d09a97fdd None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:04.111149 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 136/291] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:03:03 2022] PUT /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc/users/437efdcce5b94e21a41c9357b64beba5/roles/e71d790e22744442b6ff4af3caa1e853 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:04.190091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:04.199987 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8948a-d4e7-4093-a8db-1f2d86813979 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:04.238458 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7533372-1932-429d-8cc5-c108273e157b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:04.239136 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7533372-1932-429d-8cc5-c108273e157b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:04.239415 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7533372-1932-429d-8cc5-c108273e157b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:04.239749 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7533372-1932-429d-8cc5-c108273e157b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:04.317024 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7533372-1932-429d-8cc5-c108273e157b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:04.318558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7533372-1932-429d-8cc5-c108273e157b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:04.421689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:04.426136 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 156/292] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:04.431838 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64d8c5-f92b-4081-b4d3-6d8d09a97fdd None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:04.578409 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64d8c5-f92b-4081-b4d3-6d8d09a97fdd None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:04.625682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4949a77-f6aa-4b8c-be98-79502e377a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:04.626566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4949a77-f6aa-4b8c-be98-79502e377a4f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:04.626844 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4949a77-f6aa-4b8c-be98-79502e377a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:04.627115 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4949a77-f6aa-4b8c-be98-79502e377a4f None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:04.628407 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4949a77-f6aa-4b8c-be98-79502e377a4f None admin] RBAC: Authorizing `identity:get_project(project_id=e7099ea37d874a7aa73b15453887dffc)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:04.629864 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4949a77-f6aa-4b8c-be98-79502e377a4f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:04.631919 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 137/293] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:03:04 2022] GET /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:04.637646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7533372-1932-429d-8cc5-c108273e157b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:04.758821 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7533372-1932-429d-8cc5-c108273e157b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:04.799161 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb89fa-1450-4d8f-9d7c-d5799fe0e4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:04.799860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb89fa-1450-4d8f-9d7c-d5799fe0e4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:04.800141 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb89fa-1450-4d8f-9d7c-d5799fe0e4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:04.800418 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb89fa-1450-4d8f-9d7c-d5799fe0e4b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:04.801256 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb89fa-1450-4d8f-9d7c-d5799fe0e4b9 None admin] RBAC: Adding query filter params (role.id=b1bcb7445ee54636ae7b29a2485cc201, scope.project.id=e7099ea37d874a7aa73b15453887dffc, user.id=5864827f36c54aa89f4bc422ba9d8874) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:04.801571 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb89fa-1450-4d8f-9d7c-d5799fe0e4b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:04.803034 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb89fa-1450-4d8f-9d7c-d5799fe0e4b9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:04.809661 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 157/294] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:04 2022] GET /identity/v3/role_assignments?user.id=5864827f36c54aa89f4bc422ba9d8874&scope.project.id=e7099ea37d874a7aa73b15453887dffc&role.id=b1bcb7445ee54636ae7b29a2485cc201 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:05.386345 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918e566-6351-48c2-b39f-66a585af5eb3 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:05.386855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918e566-6351-48c2-b39f-66a585af5eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:05.386929 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918e566-6351-48c2-b39f-66a585af5eb3 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:05.388717 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 138/295] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:05.394270 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38908b-5cc2-43b6-a169-7f04e5180990 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:05.395074 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38908b-5cc2-43b6-a169-7f04e5180990 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:05.395545 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38908b-5cc2-43b6-a169-7f04e5180990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:05.416394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8f38908b-5cc2-43b6-a169-7f04e5180990 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:05.417916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8f38908b-5cc2-43b6-a169-7f04e5180990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:05.490557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:05.500197 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 158/296] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:05.522379 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918e566-6351-48c2-b39f-66a585af5eb3 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:05.553130 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:05.631767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918e566-6351-48c2-b39f-66a585af5eb3 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:05.667426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70fe2e-6381-437f-8692-05c15c1e1392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:05.668226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70fe2e-6381-437f-8692-05c15c1e1392 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:05.668510 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70fe2e-6381-437f-8692-05c15c1e1392 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:05.668784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70fe2e-6381-437f-8692-05c15c1e1392 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:05.680592 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70fe2e-6381-437f-8692-05c15c1e1392 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:05.688855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-bf70fe2e-6381-437f-8692-05c15c1e1392 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:03:05.688855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:05.688855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:05.688855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:05.688855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:05.688855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:05.688855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:05.688855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:05.688855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:05.688855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:05.688855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:05.688855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:05.688855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:05.688855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:05.688855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:05.688855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:05.689783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:05.689783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:05.689783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:05.689783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:05.689783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:05.689783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:05.689783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:05.689783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:05.689783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:05.689783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:05.689783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:05.689783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:05.689783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:05.689783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:05.689783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:05.689783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:05.690853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:05.690853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:05.690853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:05.690853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:05.690853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:05.690853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:05.690853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:05.690853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:05.690853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:05.690853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:05.690853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:03:05.690853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:05.690853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 139/297] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:05.697153 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38908b-5cc2-43b6-a169-7f04e5180990 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:05.798618 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38908b-5cc2-43b6-a169-7f04e5180990 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:05.833768 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3fd29-30c5-4aef-94e2-5fff72d549a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:05.834452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3fd29-30c5-4aef-94e2-5fff72d549a3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:05.834733 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3fd29-30c5-4aef-94e2-5fff72d549a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:05.835010 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3fd29-30c5-4aef-94e2-5fff72d549a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:05.835793 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3fd29-30c5-4aef-94e2-5fff72d549a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:05.836104 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3fd29-30c5-4aef-94e2-5fff72d549a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:05.837502 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3fd29-30c5-4aef-94e2-5fff72d549a3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:05.847216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 159/298] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:03:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:05.856705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70fe2e-6381-437f-8692-05c15c1e1392 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:05.954219 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:05.969272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70fe2e-6381-437f-8692-05c15c1e1392 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:06.006726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606838a8-8ead-46a1-93dc-9ab2cc18a611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:06.007638 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606838a8-8ead-46a1-93dc-9ab2cc18a611 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.007935 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606838a8-8ead-46a1-93dc-9ab2cc18a611 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.008211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606838a8-8ead-46a1-93dc-9ab2cc18a611 None admin] PATH_INFO: `/v3/users/2fa15e34127043dbb4646ac5a8a32903` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.009453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606838a8-8ead-46a1-93dc-9ab2cc18a611 None admin] RBAC: Authorizing `identity:get_user(user_id=2fa15e34127043dbb4646ac5a8a32903)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:06.010924 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606838a8-8ead-46a1-93dc-9ab2cc18a611 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:06.013024 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 140/299] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:03:05 2022] GET /identity/v3/users/2fa15e34127043dbb4646ac5a8a32903 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:06.018688 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3fd29-30c5-4aef-94e2-5fff72d549a3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:06.050588 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936dbf3-5a69-4efe-ab70-f762939b986b None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.051244 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936dbf3-5a69-4efe-ab70-f762939b986b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.051762 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936dbf3-5a69-4efe-ab70-f762939b986b None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.053589 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 141/300] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:06.058720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1c30d-a23c-4934-9be6-27a4281404fc None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.059240 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1c30d-a23c-4934-9be6-27a4281404fc None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.059771 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1c30d-a23c-4934-9be6-27a4281404fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.077117 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b0c1c30d-a23c-4934-9be6-27a4281404fc None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:06.078730 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b0c1c30d-a23c-4934-9be6-27a4281404fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:06.079193 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:06.138443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3fd29-30c5-4aef-94e2-5fff72d549a3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:06.163204 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 142/301] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:06.164904 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac89e7-68f9-4aa2-867e-0480378f81ac None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.165354 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac89e7-68f9-4aa2-867e-0480378f81ac None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.165648 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac89e7-68f9-4aa2-867e-0480378f81ac None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.171492 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 143/302] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:06.173671 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e6a3a-6373-49fc-a71b-c6e0f1bff734 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.173962 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e6a3a-6373-49fc-a71b-c6e0f1bff734 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.174418 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e6a3a-6373-49fc-a71b-c6e0f1bff734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.179961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717eca1e-3495-4218-a4ee-e5b60c455fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:06.181114 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717eca1e-3495-4218-a4ee-e5b60c455fc6 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.182379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717eca1e-3495-4218-a4ee-e5b60c455fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.182669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717eca1e-3495-4218-a4ee-e5b60c455fc6 None admin] PATH_INFO: `/v3/system/users/2fa15e34127043dbb4646ac5a8a32903/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.184221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717eca1e-3495-4218-a4ee-e5b60c455fc6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2fa15e34127043dbb4646ac5a8a32903, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:06.185645 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717eca1e-3495-4218-a4ee-e5b60c455fc6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:06.193944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-416e6a3a-6373-49fc-a71b-c6e0f1bff734 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:06.195203 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-416e6a3a-6373-49fc-a71b-c6e0f1bff734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:06.195632 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 160/303] 10.222.0.27 () {58 vars in 1334 bytes} [Mon Jun 27 21:03:06 2022] PUT /identity/v3/system/users/2fa15e34127043dbb4646ac5a8a32903/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:06.196975 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717eca1e-3495-4218-a4ee-e5b60c455fc6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:06.291787 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 144/304] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:03:06.315074 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717eca1e-3495-4218-a4ee-e5b60c455fc6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:06.315543 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e6a3a-6373-49fc-a71b-c6e0f1bff734 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:06.335565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:06.337365 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:06.354552 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebffcdb-1743-4f8a-8d92-90fd6612aa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:06.355387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebffcdb-1743-4f8a-8d92-90fd6612aa70 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.355700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebffcdb-1743-4f8a-8d92-90fd6612aa70 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.355976 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebffcdb-1743-4f8a-8d92-90fd6612aa70 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.370574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebffcdb-1743-4f8a-8d92-90fd6612aa70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:06.380626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-5ebffcdb-1743-4f8a-8d92-90fd6612aa70 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:03:06.380626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:06.380626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:06.380626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:06.380626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:06.380626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:06.380626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:06.380626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:06.380626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:06.380626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:06.380626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:06.380626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:06.380626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:06.380626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:06.380626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:06.380626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:06.381504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:06.381504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:06.381504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:06.381504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:06.381504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:06.381504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:06.381504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:06.381504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:06.381504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:06.381504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:06.381504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:06.381504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:06.381504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:06.381504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:06.381504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:06.381504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:06.382321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:06.382321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:06.382321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:06.382321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:06.382321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:06.382321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:06.382321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:06.382321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:06.382321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:06.382321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:06.382321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:03:06.382321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:06.382895 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 161/305] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:06.389393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebffcdb-1743-4f8a-8d92-90fd6612aa70 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:06.449705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e6a3a-6373-49fc-a71b-c6e0f1bff734 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:06.491384 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cdf7b-f810-44ca-9fcc-2b8824c802e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:06.492227 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cdf7b-f810-44ca-9fcc-2b8824c802e7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.492508 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cdf7b-f810-44ca-9fcc-2b8824c802e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.492784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cdf7b-f810-44ca-9fcc-2b8824c802e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.494008 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cdf7b-f810-44ca-9fcc-2b8824c802e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:06.495466 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cdf7b-f810-44ca-9fcc-2b8824c802e7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:06.501714 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 145/306] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:06.501714 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f24743-a5e7-4753-a9cb-ba0ee306d5d7 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.501714 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f24743-a5e7-4753-a9cb-ba0ee306d5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.501714 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f24743-a5e7-4753-a9cb-ba0ee306d5d7 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.501714 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 146/307] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:06.504586 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f24743-a5e7-4753-a9cb-ba0ee306d5d7 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:06.505229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebffcdb-1743-4f8a-8d92-90fd6612aa70 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:06.544794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472aa296-8bd5-4f57-a465-b10487334106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:06.545396 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472aa296-8bd5-4f57-a465-b10487334106 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.545681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472aa296-8bd5-4f57-a465-b10487334106 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.545960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472aa296-8bd5-4f57-a465-b10487334106 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.546713 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472aa296-8bd5-4f57-a465-b10487334106 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:06.547022 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472aa296-8bd5-4f57-a465-b10487334106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:06.548470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472aa296-8bd5-4f57-a465-b10487334106 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:06.557391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 162/308] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:03:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:06.558470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e630a-192b-4f25-8c1f-0b222525d907 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.558759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e630a-192b-4f25-8c1f-0b222525d907 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.559034 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e630a-192b-4f25-8c1f-0b222525d907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.575720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-793e630a-192b-4f25-8c1f-0b222525d907 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:06.576922 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-793e630a-192b-4f25-8c1f-0b222525d907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:06.597186 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:06.614203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:06.615529 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f24743-a5e7-4753-a9cb-ba0ee306d5d7 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:06.655628 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 163/309] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:06.655628 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e630a-192b-4f25-8c1f-0b222525d907 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:06.659788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7dc83d-7eb2-458f-bf18-08aa2eed6021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:06.660551 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dc83d-7eb2-458f-bf18-08aa2eed6021 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.660957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dc83d-7eb2-458f-bf18-08aa2eed6021 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.661310 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dc83d-7eb2-458f-bf18-08aa2eed6021 None admin] PATH_INFO: `/v3/groups` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.662134 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7dc83d-7eb2-458f-bf18-08aa2eed6021 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:06.663898 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7dc83d-7eb2-458f-bf18-08aa2eed6021 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:06.692085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 147/310] 10.222.0.27 () {60 vars in 1134 bytes} [Mon Jun 27 21:03:06 2022] POST /identity/v3/groups => generated 229 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:06.693097 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7dc83d-7eb2-458f-bf18-08aa2eed6021 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:06.792392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e630a-192b-4f25-8c1f-0b222525d907 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:06.826495 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7dc83d-7eb2-458f-bf18-08aa2eed6021 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:06.842485 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e30f1-7dd7-4c4f-b5ec-641006ca6cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:06.844016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e30f1-7dd7-4c4f-b5ec-641006ca6cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.844304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e30f1-7dd7-4c4f-b5ec-641006ca6cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.844732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e30f1-7dd7-4c4f-b5ec-641006ca6cd7 None admin] PATH_INFO: `/v3/users/bd6d5b1468b0487989f38667dd7bd09b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.859748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e30f1-7dd7-4c4f-b5ec-641006ca6cd7 None admin] RBAC: Authorizing `identity:get_user(user_id=bd6d5b1468b0487989f38667dd7bd09b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:06.861560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e30f1-7dd7-4c4f-b5ec-641006ca6cd7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:06.863493 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 164/311] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:03:06 2022] GET /identity/v3/users/bd6d5b1468b0487989f38667dd7bd09b => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:06.872573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e30f1-7dd7-4c4f-b5ec-641006ca6cd7 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:06.876983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5eb6a5-94ea-4952-a2b2-d0d74ff70a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:06.877830 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5eb6a5-94ea-4952-a2b2-d0d74ff70a82 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.878110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5eb6a5-94ea-4952-a2b2-d0d74ff70a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.878387 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5eb6a5-94ea-4952-a2b2-d0d74ff70a82 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.890501 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5eb6a5-94ea-4952-a2b2-d0d74ff70a82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:06.898430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-2d5eb6a5-94ea-4952-a2b2-d0d74ff70a82 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:03:06.898430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:06.898430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:06.898430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:06.898430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:06.898430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:06.898430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:06.898430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:06.898430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:06.898430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:06.898430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:06.898430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:06.898430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:06.898430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:06.898430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:06.898430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:06.899308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:06.899308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:06.899308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:06.899308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:06.899308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:06.899308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:06.899308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:06.899308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:06.899308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:06.899308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:06.899308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:06.899308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:06.899308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:06.899308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:06.899308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:06.899308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:06.900132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:06.900132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:06.900132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:06.900132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:06.900132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:06.900132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:06.900132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:06.900132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:06.900132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:06.900132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:06.900132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:03:06.900132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:06.900132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 148/312] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:06.905619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5eb6a5-94ea-4952-a2b2-d0d74ff70a82 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:06.910790 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:06.918060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:07.004855 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e30f1-7dd7-4c4f-b5ec-641006ca6cd7 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:07.012389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5eb6a5-94ea-4952-a2b2-d0d74ff70a82 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:07.041934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c1e01-008a-48e3-b26f-6ee684fabf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:07.043024 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c1e01-008a-48e3-b26f-6ee684fabf71 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:07.043594 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c1e01-008a-48e3-b26f-6ee684fabf71 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:07.044173 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c1e01-008a-48e3-b26f-6ee684fabf71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:07.045241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c1e01-008a-48e3-b26f-6ee684fabf71 None admin] RBAC: Adding query filter params (role.id=e71d790e22744442b6ff4af3caa1e853, user.id=bd6d5b1468b0487989f38667dd7bd09b, scope.system=all) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:07.045741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c1e01-008a-48e3-b26f-6ee684fabf71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:07.047435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c1e01-008a-48e3-b26f-6ee684fabf71 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:07.048946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94117a4-841f-437e-99ae-c9d0cbc37b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:07.049804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94117a4-841f-437e-99ae-c9d0cbc37b09 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:07.050087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94117a4-841f-437e-99ae-c9d0cbc37b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:07.050363 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94117a4-841f-437e-99ae-c9d0cbc37b09 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:07.051223 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94117a4-841f-437e-99ae-c9d0cbc37b09 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:07.051540 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94117a4-841f-437e-99ae-c9d0cbc37b09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:07.052962 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94117a4-841f-437e-99ae-c9d0cbc37b09 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:07.054620 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 165/313] 10.222.0.27 () {56 vars in 1308 bytes} [Mon Jun 27 21:03:06 2022] GET /identity/v3/role_assignments?user.id=bd6d5b1468b0487989f38667dd7bd09b&scope.system=all&role.id=e71d790e22744442b6ff4af3caa1e853 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:07.063666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 149/314] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:03:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:07.068049 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c1e01-008a-48e3-b26f-6ee684fabf71 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:07.170706 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:07.179403 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c1e01-008a-48e3-b26f-6ee684fabf71 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:07.217553 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987963c7-994f-457c-98bf-30858412ba71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:07.218841 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987963c7-994f-457c-98bf-30858412ba71 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:07.219325 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987963c7-994f-457c-98bf-30858412ba71 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:07.219854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987963c7-994f-457c-98bf-30858412ba71 None admin] PATH_INFO: `/v3/users/437efdcce5b94e21a41c9357b64beba5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:07.221463 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987963c7-994f-457c-98bf-30858412ba71 None admin] RBAC: Authorizing `identity:get_user(user_id=437efdcce5b94e21a41c9357b64beba5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:07.223259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987963c7-994f-457c-98bf-30858412ba71 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:07.225623 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 166/315] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:03:07 2022] GET /identity/v3/users/437efdcce5b94e21a41c9357b64beba5 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:07.231871 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94117a4-841f-437e-99ae-c9d0cbc37b09 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:07.307825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:07.345783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94117a4-841f-437e-99ae-c9d0cbc37b09 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:07.381777 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01810571-c29b-4766-b50e-7a1bc5d7078e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:07.382786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01810571-c29b-4766-b50e-7a1bc5d7078e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:07.383263 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01810571-c29b-4766-b50e-7a1bc5d7078e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:07.383767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01810571-c29b-4766-b50e-7a1bc5d7078e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:07.464555 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01810571-c29b-4766-b50e-7a1bc5d7078e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:07.465973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01810571-c29b-4766-b50e-7a1bc5d7078e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:07.532537 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:07.570390 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 150/316] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:07.575895 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987963c7-994f-457c-98bf-30858412ba71 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:07.681652 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987963c7-994f-457c-98bf-30858412ba71 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:07.717135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d9a69-3cb4-494e-a36d-da21e0ad7414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:07.717984 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d9a69-3cb4-494e-a36d-da21e0ad7414 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:07.718265 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d9a69-3cb4-494e-a36d-da21e0ad7414 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:07.718535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d9a69-3cb4-494e-a36d-da21e0ad7414 None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:07.719740 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d9a69-3cb4-494e-a36d-da21e0ad7414 None admin] RBAC: Authorizing `identity:get_project(project_id=e7099ea37d874a7aa73b15453887dffc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:07.721246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d9a69-3cb4-494e-a36d-da21e0ad7414 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:07.723269 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 167/317] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:03:07 2022] GET /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:07.729063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01810571-c29b-4766-b50e-7a1bc5d7078e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:07.832487 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01810571-c29b-4766-b50e-7a1bc5d7078e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:07.868140 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bcb0f-e69d-413c-bf40-1a006ead1554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:07.869149 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bcb0f-e69d-413c-bf40-1a006ead1554 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:07.869624 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bcb0f-e69d-413c-bf40-1a006ead1554 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:07.870091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bcb0f-e69d-413c-bf40-1a006ead1554 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:07.871140 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7bcb0f-e69d-413c-bf40-1a006ead1554 None admin] RBAC: Adding query filter params (role.id=e71d790e22744442b6ff4af3caa1e853, scope.project.id=e7099ea37d874a7aa73b15453887dffc, user.id=437efdcce5b94e21a41c9357b64beba5) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:07.871720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7bcb0f-e69d-413c-bf40-1a006ead1554 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:07.873380 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7bcb0f-e69d-413c-bf40-1a006ead1554 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:07.880654 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 151/318] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:07 2022] GET /identity/v3/role_assignments?user.id=437efdcce5b94e21a41c9357b64beba5&scope.project.id=e7099ea37d874a7aa73b15453887dffc&role.id=e71d790e22744442b6ff4af3caa1e853 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:08.353596 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf095f42-a334-4637-ae2c-bec02cb7a750 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:08.353861 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf095f42-a334-4637-ae2c-bec02cb7a750 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:08.354148 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf095f42-a334-4637-ae2c-bec02cb7a750 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:08.355546 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 168/319] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:08.361023 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4b76b-6999-4e70-ae53-44d12f5048d7 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:08.361317 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4b76b-6999-4e70-ae53-44d12f5048d7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:08.361635 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4b76b-6999-4e70-ae53-44d12f5048d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:08.379409 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-dec4b76b-6999-4e70-ae53-44d12f5048d7 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:08.380656 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-dec4b76b-6999-4e70-ae53-44d12f5048d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:08.408522 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:08.455848 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 152/320] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:08.478829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf095f42-a334-4637-ae2c-bec02cb7a750 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:08.510440 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:08.586418 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf095f42-a334-4637-ae2c-bec02cb7a750 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:08.621173 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c6d5d-78b1-4b1b-a938-af5c8da76ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:08.621902 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c6d5d-78b1-4b1b-a938-af5c8da76ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:08.622183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c6d5d-78b1-4b1b-a938-af5c8da76ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:08.622453 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c6d5d-78b1-4b1b-a938-af5c8da76ee7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:08.632980 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c6d5d-78b1-4b1b-a938-af5c8da76ee7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:08.640854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-bb7c6d5d-78b1-4b1b-a938-af5c8da76ee7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:03:08.640854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:08.640854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:08.640854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:08.640854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:08.640854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:08.640854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:08.640854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:08.640854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:08.640854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:08.640854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:08.640854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:08.640854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:08.640854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:08.640854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:08.640854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:08.641710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:08.641710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:08.641710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:08.641710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:08.641710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:08.641710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:08.641710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:08.641710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:08.641710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:08.641710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:08.641710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:08.641710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:08.641710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:08.641710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:08.641710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:08.641710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:08.647686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:08.647686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:08.647686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:08.647686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:08.647686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:08.647686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:08.647686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:08.647686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:08.647686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:08.647686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:08.647686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:03:08.647686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:08.647686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 169/321] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:08.648327 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4b76b-6999-4e70-ae53-44d12f5048d7 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:08.751744 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4b76b-6999-4e70-ae53-44d12f5048d7 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:08.786424 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15249fef-bb6a-4dee-ba31-9469febdad87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:08.787103 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15249fef-bb6a-4dee-ba31-9469febdad87 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:08.787388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15249fef-bb6a-4dee-ba31-9469febdad87 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:08.787688 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15249fef-bb6a-4dee-ba31-9469febdad87 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:08.788456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15249fef-bb6a-4dee-ba31-9469febdad87 None admin] RBAC: Adding query filter params (name=member) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:08.788749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15249fef-bb6a-4dee-ba31-9469febdad87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:08.790144 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15249fef-bb6a-4dee-ba31-9469febdad87 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:08.798762 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 153/322] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:03:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:08.804247 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c6d5d-78b1-4b1b-a938-af5c8da76ee7 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:08.816025 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8968b8-a17e-41f4-bf9c-7c5a4a983d10 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:08.816321 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8968b8-a17e-41f4-bf9c-7c5a4a983d10 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:08.816605 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8968b8-a17e-41f4-bf9c-7c5a4a983d10 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:08.817974 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 154/323] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:08.822921 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc229eb-02c3-4f81-b208-8e0f251e98ad None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:08.823223 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc229eb-02c3-4f81-b208-8e0f251e98ad None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:08.823506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc229eb-02c3-4f81-b208-8e0f251e98ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:08.842021 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1dc229eb-02c3-4f81-b208-8e0f251e98ad None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:08.843238 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1dc229eb-02c3-4f81-b208-8e0f251e98ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:08.861771 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:08.898479 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:08.910982 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c6d5d-78b1-4b1b-a938-af5c8da76ee7 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:08.918975 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 155/324] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:03:08.941585 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc229eb-02c3-4f81-b208-8e0f251e98ad None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:08.947818 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7c418-96f4-448c-b1f1-7f4669ee9b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:08.948756 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7c418-96f4-448c-b1f1-7f4669ee9b6f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:08.949034 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7c418-96f4-448c-b1f1-7f4669ee9b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:08.949304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7c418-96f4-448c-b1f1-7f4669ee9b6f None admin] PATH_INFO: `/v3/users/2fa15e34127043dbb4646ac5a8a32903` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:08.950506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7c418-96f4-448c-b1f1-7f4669ee9b6f None admin] RBAC: Authorizing `identity:get_user(user_id=2fa15e34127043dbb4646ac5a8a32903)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:08.952033 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7c418-96f4-448c-b1f1-7f4669ee9b6f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:08.958260 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 170/325] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:03:08 2022] GET /identity/v3/users/2fa15e34127043dbb4646ac5a8a32903 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:08.959259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7c418-96f4-448c-b1f1-7f4669ee9b6f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:09.059218 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc229eb-02c3-4f81-b208-8e0f251e98ad None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:09.077053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7c418-96f4-448c-b1f1-7f4669ee9b6f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:09.102494 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3057181-f77a-4314-a5fe-173448f13007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:09.103841 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3057181-f77a-4314-a5fe-173448f13007 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:09.104261 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3057181-f77a-4314-a5fe-173448f13007 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:09.104670 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3057181-f77a-4314-a5fe-173448f13007 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:09.106185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3057181-f77a-4314-a5fe-173448f13007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:09.107824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3057181-f77a-4314-a5fe-173448f13007 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:09.109656 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 156/326] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:09.112587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094701b-0c25-40b9-8a59-6be67c9ee733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:09.113449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094701b-0c25-40b9-8a59-6be67c9ee733 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:09.113864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094701b-0c25-40b9-8a59-6be67c9ee733 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:09.114543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094701b-0c25-40b9-8a59-6be67c9ee733 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:09.115323 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3057181-f77a-4314-a5fe-173448f13007 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:09.115730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094701b-0c25-40b9-8a59-6be67c9ee733 None admin] RBAC: Adding query filter params (role.id=b1bcb7445ee54636ae7b29a2485cc201, user.id=2fa15e34127043dbb4646ac5a8a32903, scope.system=all) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:09.116177 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094701b-0c25-40b9-8a59-6be67c9ee733 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:09.117739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094701b-0c25-40b9-8a59-6be67c9ee733 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:09.131881 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 171/327] 10.222.0.27 () {56 vars in 1308 bytes} [Mon Jun 27 21:03:08 2022] GET /identity/v3/role_assignments?user.id=2fa15e34127043dbb4646ac5a8a32903&scope.system=all&role.id=b1bcb7445ee54636ae7b29a2485cc201 => generated 510 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:09.138582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:09.181137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c75175-ba23-4ffd-8088-5eb95c267c81 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:09.181448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c75175-ba23-4ffd-8088-5eb95c267c81 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:09.181754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c75175-ba23-4ffd-8088-5eb95c267c81 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:09.183398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 172/328] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:09.189399 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12b23d-7b76-48e3-9586-e1eadbbaeaef None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:09.190125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12b23d-7b76-48e3-9586-e1eadbbaeaef None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:09.190507 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12b23d-7b76-48e3-9586-e1eadbbaeaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:09.208459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8a12b23d-7b76-48e3-9586-e1eadbbaeaef None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:09.209461 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8a12b23d-7b76-48e3-9586-e1eadbbaeaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:09.222628 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:09.229550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3057181-f77a-4314-a5fe-173448f13007 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:09.268768 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb061a-6562-4a0d-a40d-ed51fef56f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:09.269456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb061a-6562-4a0d-a40d-ed51fef56f14 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:09.269734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb061a-6562-4a0d-a40d-ed51fef56f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:09.270001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb061a-6562-4a0d-a40d-ed51fef56f14 None admin] PATH_INFO: `/v3/groups` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:09.271229 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb061a-6562-4a0d-a40d-ed51fef56f14 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:09.272750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb061a-6562-4a0d-a40d-ed51fef56f14 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:09.285897 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 173/329] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:03:09.295106 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 157/330] 10.222.0.27 () {60 vars in 1134 bytes} [Mon Jun 27 21:03:09 2022] POST /identity/v3/groups => generated 226 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:03:09.309698 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12b23d-7b76-48e3-9586-e1eadbbaeaef None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:09.420432 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12b23d-7b76-48e3-9586-e1eadbbaeaef None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:09.457301 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c724d8a-7171-4609-aa76-469d64b396e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:09.458011 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c724d8a-7171-4609-aa76-469d64b396e7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:09.458287 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c724d8a-7171-4609-aa76-469d64b396e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:09.458553 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c724d8a-7171-4609-aa76-469d64b396e7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:09.468634 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c724d8a-7171-4609-aa76-469d64b396e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:09.476476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-9c724d8a-7171-4609-aa76-469d64b396e7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:03:09.476476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:09.476476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:09.476476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:09.476476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:09.476476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:09.476476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:09.476476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:09.476476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:09.476476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:09.476476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:09.476476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:09.476476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:09.476476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:09.476476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:09.476476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:09.477526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:09.477526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:09.477526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:09.477526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:09.477526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:09.477526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:09.477526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:09.477526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:09.477526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:09.477526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:09.477526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:09.477526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:09.477526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:09.477526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:09.477526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:09.477526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:09.478514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:09.478514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:09.478514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:09.478514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:09.478514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:09.478514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:09.478514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:09.478514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:09.478514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:09.478514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:09.478514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:03:09.478514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:09.478514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 174/331] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:09.483791 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb061a-6562-4a0d-a40d-ed51fef56f14 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:09.570447 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:09.594004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb061a-6562-4a0d-a40d-ed51fef56f14 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:09.629539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ec945-cf76-460a-bec6-4ef0d6879d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:09.630223 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ec945-cf76-460a-bec6-4ef0d6879d6a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:09.630504 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ec945-cf76-460a-bec6-4ef0d6879d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:09.630780 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ec945-cf76-460a-bec6-4ef0d6879d6a None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:09.631535 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ec945-cf76-460a-bec6-4ef0d6879d6a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:09.631887 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ec945-cf76-460a-bec6-4ef0d6879d6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:09.633344 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ec945-cf76-460a-bec6-4ef0d6879d6a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:09.642774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 158/332] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:03:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:09.648433 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c724d8a-7171-4609-aa76-469d64b396e7 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:09.660356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:09.760333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c724d8a-7171-4609-aa76-469d64b396e7 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:09.794410 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a2954-44ac-4eee-8426-d726bb86b92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:09.795313 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a2954-44ac-4eee-8426-d726bb86b92e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:09.795594 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a2954-44ac-4eee-8426-d726bb86b92e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:09.795895 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a2954-44ac-4eee-8426-d726bb86b92e None admin] PATH_INFO: `/v3/users/bd6d5b1468b0487989f38667dd7bd09b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:09.797087 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a2954-44ac-4eee-8426-d726bb86b92e None admin] RBAC: Authorizing `identity:get_user(user_id=bd6d5b1468b0487989f38667dd7bd09b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:09.798568 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a2954-44ac-4eee-8426-d726bb86b92e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:09.800567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 175/333] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:03:09 2022] GET /identity/v3/users/bd6d5b1468b0487989f38667dd7bd09b => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:09.806008 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ec945-cf76-460a-bec6-4ef0d6879d6a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:09.921838 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ec945-cf76-460a-bec6-4ef0d6879d6a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:09.965728 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e1700-12ee-4428-819e-7a041572975e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:09.966630 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e1700-12ee-4428-819e-7a041572975e None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:09.966916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e1700-12ee-4428-819e-7a041572975e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:09.967197 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e1700-12ee-4428-819e-7a041572975e None admin] PATH_INFO: `/v3/system/users/bd6d5b1468b0487989f38667dd7bd09b/roles/e71d790e22744442b6ff4af3caa1e853` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:09.970132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e1700-12ee-4428-819e-7a041572975e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bd6d5b1468b0487989f38667dd7bd09b, role_id=e71d790e22744442b6ff4af3caa1e853)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:09.971698 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e1700-12ee-4428-819e-7a041572975e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:09.988865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 159/334] 10.222.0.27 () {58 vars in 1334 bytes} [Mon Jun 27 21:03:09 2022] PUT /identity/v3/system/users/bd6d5b1468b0487989f38667dd7bd09b/roles/e71d790e22744442b6ff4af3caa1e853 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:11.526731 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fda60f-ee5c-4af3-b58b-54fa4fee216c None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:11.527247 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fda60f-ee5c-4af3-b58b-54fa4fee216c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:11.527322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fda60f-ee5c-4af3-b58b-54fa4fee216c None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:11.528988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 176/335] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:11.534678 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b38c7-7831-4ebc-996d-4656c39f99c2 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:11.535111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b38c7-7831-4ebc-996d-4656c39f99c2 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:11.535279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b38c7-7831-4ebc-996d-4656c39f99c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:11.556792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:11.562440 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d40b38c7-7831-4ebc-996d-4656c39f99c2 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:11.563693 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d40b38c7-7831-4ebc-996d-4656c39f99c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:11.653763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 160/336] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:03:11.692627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fda60f-ee5c-4af3-b58b-54fa4fee216c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:11.793233 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:11.824767 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fda60f-ee5c-4af3-b58b-54fa4fee216c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:11.863601 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e78009-ce55-4294-8631-58b6c7d245d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:11.864802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e78009-ce55-4294-8631-58b6c7d245d2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:11.865272 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e78009-ce55-4294-8631-58b6c7d245d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:11.865732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e78009-ce55-4294-8631-58b6c7d245d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:11.878160 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e78009-ce55-4294-8631-58b6c7d245d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:11.886198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-d6e78009-ce55-4294-8631-58b6c7d245d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:03:11.886198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:11.886198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:11.886198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:11.886198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:11.886198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:11.886198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:11.886198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:11.886198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:11.886198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:11.886198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:11.886198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:11.886198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:11.886198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:11.886198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:11.886198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:11.887053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:11.887053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:11.887053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:11.887053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:11.887053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:11.887053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:11.887053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:11.887053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:11.887053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:11.887053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:11.887053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:11.887053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:11.887053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:11.887053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:11.887053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:11.887053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:11.887862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:11.887862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:11.887862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:11.887862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:11.887862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:11.887862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:11.887862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:11.887862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:11.887862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:11.887862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:11.887862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:03:11.887862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:11.890215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 177/337] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:11.896748 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b38c7-7831-4ebc-996d-4656c39f99c2 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:12.024506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c6fb2-29ce-45c9-8f2a-bb6067493d75 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.025207 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c6fb2-29ce-45c9-8f2a-bb6067493d75 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.025783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c6fb2-29ce-45c9-8f2a-bb6067493d75 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:12.027357 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 178/338] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:12.031834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b38c7-7831-4ebc-996d-4656c39f99c2 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:12.053978 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3a978-5777-43e8-8d14-b1ad806df0f9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.054649 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3a978-5777-43e8-8d14-b1ad806df0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.055127 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3a978-5777-43e8-8d14-b1ad806df0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:12.081246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-bef3a978-5777-43e8-8d14-b1ad806df0f9 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:12.082803 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bef3a978-5777-43e8-8d14-b1ad806df0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:12.085349 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2d68c-252c-4c5b-98c1-467da8ffc91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:12.086265 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2d68c-252c-4c5b-98c1-467da8ffc91c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.086714 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2d68c-252c-4c5b-98c1-467da8ffc91c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.087125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2d68c-252c-4c5b-98c1-467da8ffc91c None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:12.088474 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2d68c-252c-4c5b-98c1-467da8ffc91c None admin] RBAC: Adding query filter params (name=member) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:12.088923 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2d68c-252c-4c5b-98c1-467da8ffc91c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:12.090486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2d68c-252c-4c5b-98c1-467da8ffc91c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:12.093364 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:12.109048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 161/339] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:03:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:12.114505 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2d68c-252c-4c5b-98c1-467da8ffc91c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:12.195729 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 179/340] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:03:12.238853 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3a978-5777-43e8-8d14-b1ad806df0f9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:12.301574 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2d68c-252c-4c5b-98c1-467da8ffc91c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:12.346178 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dda7f7-a89d-4996-8fe1-be77af273761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:12.346766 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dda7f7-a89d-4996-8fe1-be77af273761 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.347052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dda7f7-a89d-4996-8fe1-be77af273761 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.347329 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dda7f7-a89d-4996-8fe1-be77af273761 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:12.363914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:12.408784 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3a978-5777-43e8-8d14-b1ad806df0f9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:12.447753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43c397-a381-42a0-a44c-d6e02ee66343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:12.448572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43c397-a381-42a0-a44c-d6e02ee66343 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.449007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43c397-a381-42a0-a44c-d6e02ee66343 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.449443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43c397-a381-42a0-a44c-d6e02ee66343 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:12.461272 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43c397-a381-42a0-a44c-d6e02ee66343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:12.469678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-0d43c397-a381-42a0-a44c-d6e02ee66343 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:03:12.469678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:12.469678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:12.469678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:12.469678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:12.469678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:12.469678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:12.469678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:12.469678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:12.469678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:12.469678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:12.469678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:12.469678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:12.469678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:12.469678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:12.469678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:12.470703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:12.470703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:12.470703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:12.470703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:12.470703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:12.470703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:12.470703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:12.470703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:12.470703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:12.470703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:12.470703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:12.470703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:12.470703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:12.470703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:12.470703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:12.470703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:12.471739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:12.471739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:12.471739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:12.471739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:12.471739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:12.471739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:12.471739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:12.471739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:12.471739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:12.471739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:12.471739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:03:12.471739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:12.474038 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 180/341] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:03:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:12.475411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c419889-a1cd-43e5-aebb-b22595a9085f None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.475867 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c419889-a1cd-43e5-aebb-b22595a9085f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.478190 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c419889-a1cd-43e5-aebb-b22595a9085f None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:12.479989 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 181/342] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:12.484910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c419889-a1cd-43e5-aebb-b22595a9085f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:12.490105 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:12.500941 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dda7f7-a89d-4996-8fe1-be77af273761 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:12.502743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dda7f7-a89d-4996-8fe1-be77af273761 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:12.595445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c419889-a1cd-43e5-aebb-b22595a9085f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:12.621328 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:12.633576 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b188227-a289-4966-b52f-a9ea0fe7a20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:12.634470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b188227-a289-4966-b52f-a9ea0fe7a20a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.634910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b188227-a289-4966-b52f-a9ea0fe7a20a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.635507 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b188227-a289-4966-b52f-a9ea0fe7a20a None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:12.636408 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b188227-a289-4966-b52f-a9ea0fe7a20a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:12.636855 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b188227-a289-4966-b52f-a9ea0fe7a20a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:12.638369 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b188227-a289-4966-b52f-a9ea0fe7a20a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:12.647391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 182/343] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:12.648899 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a3585-5d55-4b82-ad00-4dcbcc80adbd None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.649322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a3585-5d55-4b82-ad00-4dcbcc80adbd None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.649788 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a3585-5d55-4b82-ad00-4dcbcc80adbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:12.667318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-026a3585-5d55-4b82-ad00-4dcbcc80adbd None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:12.668769 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-026a3585-5d55-4b82-ad00-4dcbcc80adbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:12.723636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 162/344] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 21:03:12.725269 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dda7f7-a89d-4996-8fe1-be77af273761 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:12.742379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 183/345] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:12.743312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a3585-5d55-4b82-ad00-4dcbcc80adbd None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:12.850176 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a3585-5d55-4b82-ad00-4dcbcc80adbd None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:12.869033 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:12.871176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dda7f7-a89d-4996-8fe1-be77af273761 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:12.877623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:12.892879 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84e2bb-235e-4d03-9e08-17482f6d40a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:12.894037 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84e2bb-235e-4d03-9e08-17482f6d40a7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.894552 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84e2bb-235e-4d03-9e08-17482f6d40a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.894987 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84e2bb-235e-4d03-9e08-17482f6d40a7 None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:12.896431 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84e2bb-235e-4d03-9e08-17482f6d40a7 None admin] RBAC: Authorizing `identity:get_project(project_id=0dc9df6fada54f72a35899fa21f03b20)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:12.898214 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84e2bb-235e-4d03-9e08-17482f6d40a7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:12.901064 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 184/346] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:03:12 2022] GET /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:12.902124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84e2bb-235e-4d03-9e08-17482f6d40a7 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:12.939198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5912d-7fa8-4935-b300-9656a6d278f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:12.940108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5912d-7fa8-4935-b300-9656a6d278f4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.940462 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5912d-7fa8-4935-b300-9656a6d278f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.940963 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5912d-7fa8-4935-b300-9656a6d278f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.025491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d5912d-7fa8-4935-b300-9656a6d278f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:13.026989 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84e2bb-235e-4d03-9e08-17482f6d40a7 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:13.027549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d5912d-7fa8-4935-b300-9656a6d278f4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:13.088355 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741795b-0659-4f46-99b8-8beb2a1f8996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:13.089185 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741795b-0659-4f46-99b8-8beb2a1f8996 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:13.089467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741795b-0659-4f46-99b8-8beb2a1f8996 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:13.089744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741795b-0659-4f46-99b8-8beb2a1f8996 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.104587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741795b-0659-4f46-99b8-8beb2a1f8996 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:13.116820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-0741795b-0659-4f46-99b8-8beb2a1f8996 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:03:13.116820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:13.116820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:13.116820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:13.116820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:13.116820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:13.116820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:13.116820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:13.116820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:13.116820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:13.116820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:13.116820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:13.116820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:13.116820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:13.116820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:13.116820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:13.118091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:13.118091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:13.118091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:13.118091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:13.118091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:13.118091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:13.118091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:13.118091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:13.118091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:13.118091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:13.118091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:13.118091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:13.118091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:13.118091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:13.118091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:13.118091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:13.119098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:13.119098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:13.119098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:13.119098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:13.119098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:13.119098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:13.119098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:13.119098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:13.119098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:13.119098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:13.119098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:03:13.119098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:13.121229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 185/347] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:13.125743 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741795b-0659-4f46-99b8-8beb2a1f8996 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:13.136543 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:13.146937 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 163/348] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:13.147828 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5912d-7fa8-4935-b300-9656a6d278f4 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:13.198798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:13.258566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741795b-0659-4f46-99b8-8beb2a1f8996 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:13.299044 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252b2bb-b136-4d48-ab88-f7337ec3d780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:13.300317 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252b2bb-b136-4d48-ab88-f7337ec3d780 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:13.300780 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252b2bb-b136-4d48-ab88-f7337ec3d780 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:13.301256 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252b2bb-b136-4d48-ab88-f7337ec3d780 None admin] PATH_INFO: `/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.302735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5912d-7fa8-4935-b300-9656a6d278f4 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:13.310777 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252b2bb-b136-4d48-ab88-f7337ec3d780 None admin] RBAC: Authorizing `identity:get_group(group_id=04f5c67e865c4a30a1bd97093e65cbcd)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:13.312292 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252b2bb-b136-4d48-ab88-f7337ec3d780 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:13.314338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 186/349] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Jun 27 21:03:13 2022] GET /identity/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:13.317910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252b2bb-b136-4d48-ab88-f7337ec3d780 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:13.353639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56297b-506c-40fb-b16d-72e44a462720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:13.354592 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56297b-506c-40fb-b16d-72e44a462720 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:13.355021 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56297b-506c-40fb-b16d-72e44a462720 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:13.355442 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56297b-506c-40fb-b16d-72e44a462720 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.356380 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56297b-506c-40fb-b16d-72e44a462720 None admin] RBAC: Adding query filter params (name=member) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:13.356833 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56297b-506c-40fb-b16d-72e44a462720 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:13.358405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56297b-506c-40fb-b16d-72e44a462720 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:13.368342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 164/350] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:03:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:13.369831 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08856dff-ba18-4008-afb6-c4ab1076ef77 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:13.370252 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08856dff-ba18-4008-afb6-c4ab1076ef77 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:13.370660 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08856dff-ba18-4008-afb6-c4ab1076ef77 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.372375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 165/351] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:13.374613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac275c-4356-4af5-add4-1f2c2bc4b779 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:13.375032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac275c-4356-4af5-add4-1f2c2bc4b779 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:13.375435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac275c-4356-4af5-add4-1f2c2bc4b779 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.381085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 166/352] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:13.382393 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac275c-4356-4af5-add4-1f2c2bc4b779 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:13.450320 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252b2bb-b136-4d48-ab88-f7337ec3d780 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:13.458322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:13.462054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:13.501250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffce64f-13ca-4cb8-8559-139b34628503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:13.502157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffce64f-13ca-4cb8-8559-139b34628503 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:13.502435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffce64f-13ca-4cb8-8559-139b34628503 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:13.502706 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffce64f-13ca-4cb8-8559-139b34628503 None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.503976 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffce64f-13ca-4cb8-8559-139b34628503 None admin] RBAC: Authorizing `identity:get_project(project_id=0dc9df6fada54f72a35899fa21f03b20)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:13.505388 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffce64f-13ca-4cb8-8559-139b34628503 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:13.507105 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 187/353] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:03:13 2022] GET /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:13.510340 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffce64f-13ca-4cb8-8559-139b34628503 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:13.514132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac275c-4356-4af5-add4-1f2c2bc4b779 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:13.551960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c9d85-87d7-461c-a94a-034e5ad87bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:13.552729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c9d85-87d7-461c-a94a-034e5ad87bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:13.553007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c9d85-87d7-461c-a94a-034e5ad87bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:13.553917 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c9d85-87d7-461c-a94a-034e5ad87bc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.554115 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c9d85-87d7-461c-a94a-034e5ad87bc2 None admin] RBAC: Adding query filter params (group.id=04f5c67e865c4a30a1bd97093e65cbcd, role.id=b1bcb7445ee54636ae7b29a2485cc201, scope.project.id=0dc9df6fada54f72a35899fa21f03b20) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:13.554413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c9d85-87d7-461c-a94a-034e5ad87bc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:13.555975 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c9d85-87d7-461c-a94a-034e5ad87bc2 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:13.563213 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 167/354] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Jun 27 21:03:13 2022] GET /identity/v3/role_assignments?group.id=04f5c67e865c4a30a1bd97093e65cbcd&scope.project.id=0dc9df6fada54f72a35899fa21f03b20&role.id=b1bcb7445ee54636ae7b29a2485cc201 => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:13.564513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b714c-cee7-4bcd-b3d4-0ec2f554879d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:13.564836 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b714c-cee7-4bcd-b3d4-0ec2f554879d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:13.565110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b714c-cee7-4bcd-b3d4-0ec2f554879d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.583619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-495b714c-cee7-4bcd-b3d4-0ec2f554879d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:13.584890 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-495b714c-cee7-4bcd-b3d4-0ec2f554879d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:13.647426 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffce64f-13ca-4cb8-8559-139b34628503 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:13.661759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 168/355] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:13.662970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c0e17-0c0a-420a-b8e0-b0ce5e1cffe7 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:13.663269 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c0e17-0c0a-420a-b8e0-b0ce5e1cffe7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:13.667989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c0e17-0c0a-420a-b8e0-b0ce5e1cffe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.696410 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b588cf3-8197-4393-8153-fc95dbcafe04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:13.703716 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b588cf3-8197-4393-8153-fc95dbcafe04 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:13.703716 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b588cf3-8197-4393-8153-fc95dbcafe04 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:13.703716 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b588cf3-8197-4393-8153-fc95dbcafe04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.703995 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d35c0e17-0c0a-420a-b8e0-b0ce5e1cffe7 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:13.703995 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d35c0e17-0c0a-420a-b8e0-b0ce5e1cffe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:13.763355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:13.765020 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:13.790838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b588cf3-8197-4393-8153-fc95dbcafe04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:13.793324 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b588cf3-8197-4393-8153-fc95dbcafe04 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:13.794375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 169/356] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:13.795513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c0e17-0c0a-420a-b8e0-b0ce5e1cffe7 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:13.898206 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 188/357] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:13.899169 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b588cf3-8197-4393-8153-fc95dbcafe04 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:13.900820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c0e17-0c0a-420a-b8e0-b0ce5e1cffe7 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:13.939801 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5cafa-a4b2-4a6d-ba44-902310557f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:13.941073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5cafa-a4b2-4a6d-ba44-902310557f6c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:13.941565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5cafa-a4b2-4a6d-ba44-902310557f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:13.942037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5cafa-a4b2-4a6d-ba44-902310557f6c None admin] PATH_INFO: `/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.943435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a5cafa-a4b2-4a6d-ba44-902310557f6c None admin] RBAC: Authorizing `identity:get_group(group_id=04f5c67e865c4a30a1bd97093e65cbcd)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:13.945121 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a5cafa-a4b2-4a6d-ba44-902310557f6c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:13.947076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 170/358] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Jun 27 21:03:13 2022] GET /identity/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:13.954024 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5cafa-a4b2-4a6d-ba44-902310557f6c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:13.981015 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:14.007699 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b588cf3-8197-4393-8153-fc95dbcafe04 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:14.014016 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:14.050114 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341d37b-81aa-4bc2-9c32-16f4be064523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:14.051022 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341d37b-81aa-4bc2-9c32-16f4be064523 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:14.051305 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341d37b-81aa-4bc2-9c32-16f4be064523 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:14.051578 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341d37b-81aa-4bc2-9c32-16f4be064523 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:14.063411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341d37b-81aa-4bc2-9c32-16f4be064523 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:14.071747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-9341d37b-81aa-4bc2-9c32-16f4be064523 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:03:14.071747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:14.071747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:14.071747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:14.071747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:14.071747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:14.071747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:14.071747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:14.071747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:14.071747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:14.071747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:14.071747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:14.071747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:14.071747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:14.071747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:14.071747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:14.072607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:14.072607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:14.072607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:14.072607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:14.072607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:14.072607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:14.072607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:14.072607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:14.072607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:14.072607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:14.072607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:14.072607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:14.072607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:14.072607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:14.072607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:14.072607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:14.073389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:14.073389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:14.073389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:14.073389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:14.073389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:14.073389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:14.073389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:14.073389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:14.073389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:14.073389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:14.073389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:03:14.073389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:14.074095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 189/359] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:03:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:14.074683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5cafa-a4b2-4a6d-ba44-902310557f6c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:14.074795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341d37b-81aa-4bc2-9c32-16f4be064523 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:14.119124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3d851-9e61-4d6e-82d1-28c70c47d4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:14.120164 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3d851-9e61-4d6e-82d1-28c70c47d4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:14.120518 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3d851-9e61-4d6e-82d1-28c70c47d4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:14.120867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3d851-9e61-4d6e-82d1-28c70c47d4b5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:14.140639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3d851-9e61-4d6e-82d1-28c70c47d4b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:14.149153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-1fe3d851-9e61-4d6e-82d1-28c70c47d4b5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:03:14.149153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:14.149153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:14.149153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:14.149153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:14.149153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:14.149153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:14.149153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:14.149153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:14.149153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:14.149153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:14.149153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:14.149153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:14.149153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:14.149153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:14.149153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:14.150306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:14.150306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:14.150306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:14.150306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:14.150306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:14.150306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:14.150306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:14.150306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:14.150306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:14.150306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:14.150306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:14.150306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:14.150306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:14.150306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:14.150306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:14.150306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:14.151486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:14.151486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:14.151486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:14.151486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:14.151486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:14.151486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:14.151486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:14.151486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:14.151486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:14.151486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:14.151486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:03:14.151486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:14.151486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 171/360] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:14.151486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3d851-9e61-4d6e-82d1-28c70c47d4b5 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:14.208609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341d37b-81aa-4bc2-9c32-16f4be064523 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:14.252585 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb75f9b-77bb-4488-a5e1-bae3fc0a9bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:14.253806 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb75f9b-77bb-4488-a5e1-bae3fc0a9bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:14.254274 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb75f9b-77bb-4488-a5e1-bae3fc0a9bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:14.254737 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb75f9b-77bb-4488-a5e1-bae3fc0a9bb9 None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:14.256240 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb75f9b-77bb-4488-a5e1-bae3fc0a9bb9 None admin] RBAC: Authorizing `identity:get_project(project_id=e7099ea37d874a7aa73b15453887dffc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:14.257901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb75f9b-77bb-4488-a5e1-bae3fc0a9bb9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:14.260592 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 190/361] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:03:14 2022] GET /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:14.262116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb470226-4538-4897-a081-9007903e87ad None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:14.262572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb470226-4538-4897-a081-9007903e87ad None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:14.263031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb470226-4538-4897-a081-9007903e87ad None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:14.264482 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 191/362] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:14.265682 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb470226-4538-4897-a081-9007903e87ad None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:14.270332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:14.294033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3d851-9e61-4d6e-82d1-28c70c47d4b5 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:14.310589 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:14.340148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69e7bf-eb71-449a-bd25-5ebbbf599343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:14.340910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69e7bf-eb71-449a-bd25-5ebbbf599343 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:14.341244 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69e7bf-eb71-449a-bd25-5ebbbf599343 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:14.341523 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69e7bf-eb71-449a-bd25-5ebbbf599343 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:14.342365 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff69e7bf-eb71-449a-bd25-5ebbbf599343 None admin] RBAC: Adding query filter params (group.id=04f5c67e865c4a30a1bd97093e65cbcd, role.id=b3efb2b2e43c4af8b1cf36a37f1e05f9, scope.project.id=0dc9df6fada54f72a35899fa21f03b20) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:14.342681 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff69e7bf-eb71-449a-bd25-5ebbbf599343 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:14.344210 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff69e7bf-eb71-449a-bd25-5ebbbf599343 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:14.350400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 172/363] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Jun 27 21:03:14 2022] GET /identity/v3/role_assignments?group.id=04f5c67e865c4a30a1bd97093e65cbcd&scope.project.id=0dc9df6fada54f72a35899fa21f03b20&role.id=b3efb2b2e43c4af8b1cf36a37f1e05f9 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:14.350969 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69e7bf-eb71-449a-bd25-5ebbbf599343 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:14.392470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb470226-4538-4897-a081-9007903e87ad None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:14.435514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca46cb0-c234-4b2e-906f-e4f8b1d844bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:14.438711 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca46cb0-c234-4b2e-906f-e4f8b1d844bf None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:14.439078 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca46cb0-c234-4b2e-906f-e4f8b1d844bf None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:14.439363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca46cb0-c234-4b2e-906f-e4f8b1d844bf None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:14.440321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca46cb0-c234-4b2e-906f-e4f8b1d844bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:14.440637 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca46cb0-c234-4b2e-906f-e4f8b1d844bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:14.442137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca46cb0-c234-4b2e-906f-e4f8b1d844bf None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:14.452891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 192/364] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:14.453882 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca46cb0-c234-4b2e-906f-e4f8b1d844bf None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:14.474563 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69e7bf-eb71-449a-bd25-5ebbbf599343 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:14.522081 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38309432-bcae-48d9-8b8a-d8d1fc0cc3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:14.522767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38309432-bcae-48d9-8b8a-d8d1fc0cc3de None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:14.523045 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38309432-bcae-48d9-8b8a-d8d1fc0cc3de None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:14.523320 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38309432-bcae-48d9-8b8a-d8d1fc0cc3de None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:14.524110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38309432-bcae-48d9-8b8a-d8d1fc0cc3de None admin] RBAC: Adding query filter params (name=reader) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:14.524416 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38309432-bcae-48d9-8b8a-d8d1fc0cc3de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:14.525807 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38309432-bcae-48d9-8b8a-d8d1fc0cc3de None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:14.538520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 173/365] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:03:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:14.543192 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2b0c5-3813-4353-895e-830a76c961cf None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:14.543192 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2b0c5-3813-4353-895e-830a76c961cf None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:14.543192 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2b0c5-3813-4353-895e-830a76c961cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:14.550696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:14.560115 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-bff2b0c5-3813-4353-895e-830a76c961cf None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:14.561321 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-bff2b0c5-3813-4353-895e-830a76c961cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:14.595646 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:14.598455 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca46cb0-c234-4b2e-906f-e4f8b1d844bf None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:14.640448 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 174/366] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:14.641745 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2b0c5-3813-4353-895e-830a76c961cf None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:14.647479 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087ba32-f5a3-41f0-84dd-106dd350c7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:14.648953 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087ba32-f5a3-41f0-84dd-106dd350c7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:14.649409 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087ba32-f5a3-41f0-84dd-106dd350c7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:14.649876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087ba32-f5a3-41f0-84dd-106dd350c7e3 None admin] PATH_INFO: `/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:14.651817 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087ba32-f5a3-41f0-84dd-106dd350c7e3 None admin] RBAC: Authorizing `identity:get_group(group_id=04f5c67e865c4a30a1bd97093e65cbcd)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:14.653472 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087ba32-f5a3-41f0-84dd-106dd350c7e3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:14.655798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 193/367] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Jun 27 21:03:14 2022] GET /identity/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:14.656721 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087ba32-f5a3-41f0-84dd-106dd350c7e3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:14.781351 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2b0c5-3813-4353-895e-830a76c961cf None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:14.792405 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087ba32-f5a3-41f0-84dd-106dd350c7e3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:14.827152 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0b066-9212-451b-85ec-c276c11bd9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:14.828211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0b066-9212-451b-85ec-c276c11bd9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:14.828800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0b066-9212-451b-85ec-c276c11bd9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:14.829341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0b066-9212-451b-85ec-c276c11bd9a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:14.830207 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01440477-d83c-4296-9223-5a3e64ed620e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:14.831193 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01440477-d83c-4296-9223-5a3e64ed620e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:14.831555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01440477-d83c-4296-9223-5a3e64ed620e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:14.831935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01440477-d83c-4296-9223-5a3e64ed620e None admin] PATH_INFO: `/v3/users/bd6d5b1468b0487989f38667dd7bd09b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:14.833230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01440477-d83c-4296-9223-5a3e64ed620e None admin] RBAC: Authorizing `identity:get_user(user_id=bd6d5b1468b0487989f38667dd7bd09b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:14.834990 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01440477-d83c-4296-9223-5a3e64ed620e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:14.836513 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 194/368] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:03:14 2022] GET /identity/v3/users/bd6d5b1468b0487989f38667dd7bd09b => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:14.837687 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01440477-d83c-4296-9223-5a3e64ed620e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:14.840470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:14.874528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:14.913366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0b066-9212-451b-85ec-c276c11bd9a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:14.915250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0b066-9212-451b-85ec-c276c11bd9a1 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:14.959803 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01440477-d83c-4296-9223-5a3e64ed620e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:15.002779 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf4156-a0fc-4dad-a76e-3c01b96dd1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:15.003456 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf4156-a0fc-4dad-a76e-3c01b96dd1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:15.004026 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf4156-a0fc-4dad-a76e-3c01b96dd1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:15.004146 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf4156-a0fc-4dad-a76e-3c01b96dd1a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:15.005018 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf4156-a0fc-4dad-a76e-3c01b96dd1a8 None admin] RBAC: Adding query filter params (group.id=04f5c67e865c4a30a1bd97093e65cbcd, role.id=b1bcb7445ee54636ae7b29a2485cc201, scope.project.id=e7099ea37d874a7aa73b15453887dffc) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:15.005246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf4156-a0fc-4dad-a76e-3c01b96dd1a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:15.006829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf4156-a0fc-4dad-a76e-3c01b96dd1a8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:15.013176 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 195/369] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Jun 27 21:03:14 2022] GET /identity/v3/role_assignments?group.id=04f5c67e865c4a30a1bd97093e65cbcd&scope.project.id=e7099ea37d874a7aa73b15453887dffc&role.id=b1bcb7445ee54636ae7b29a2485cc201 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:15.018301 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf4156-a0fc-4dad-a76e-3c01b96dd1a8 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:15.052007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 175/370] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:15.053195 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0b066-9212-451b-85ec-c276c11bd9a1 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:15.127832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:15.175284 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:15.203177 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf4156-a0fc-4dad-a76e-3c01b96dd1a8 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:15.211430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0b066-9212-451b-85ec-c276c11bd9a1 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:15.244008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714186a-cc00-47e3-8a85-bddddce6b17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:15.245009 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714186a-cc00-47e3-8a85-bddddce6b17e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:15.245291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714186a-cc00-47e3-8a85-bddddce6b17e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:15.245565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714186a-cc00-47e3-8a85-bddddce6b17e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:15.260003 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5714186a-cc00-47e3-8a85-bddddce6b17e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:15.271720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-5714186a-cc00-47e3-8a85-bddddce6b17e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:15.271720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:15.271720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:15.271720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:15.271720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:15.271720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:15.271720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:15.271720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:15.271720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:15.271720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:15.271720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:15.271720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:15.271720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:15.271720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:15.271720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:15.271720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:15.272712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:15.272712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:15.272712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:15.272712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:15.272712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:15.272712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:15.272712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:15.272712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:15.272712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:15.272712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:15.272712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:15.272712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:15.272712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:15.272712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:15.272712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:15.272712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:15.273525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:15.273525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:15.273525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:15.273525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:15.273525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:15.273525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:15.273525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:15.273525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:15.273525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:15.273525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:15.273525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:15.273525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:15.274172 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 196/371] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:15.275842 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714186a-cc00-47e3-8a85-bddddce6b17e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:15.323835 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d772e-3a62-4ae6-9164-7e85e289df3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:15.324428 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d772e-3a62-4ae6-9164-7e85e289df3f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:15.324708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d772e-3a62-4ae6-9164-7e85e289df3f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:15.324980 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d772e-3a62-4ae6-9164-7e85e289df3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:15.325825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d772e-3a62-4ae6-9164-7e85e289df3f None admin] RBAC: Adding query filter params (role.id=e71d790e22744442b6ff4af3caa1e853, user.id=bd6d5b1468b0487989f38667dd7bd09b, scope.system=all) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:15.326134 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d772e-3a62-4ae6-9164-7e85e289df3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:15.327579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d772e-3a62-4ae6-9164-7e85e289df3f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:15.348756 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 176/372] 10.222.0.27 () {56 vars in 1308 bytes} [Mon Jun 27 21:03:15 2022] GET /identity/v3/role_assignments?user.id=bd6d5b1468b0487989f38667dd7bd09b&scope.system=all&role.id=e71d790e22744442b6ff4af3caa1e853 => generated 510 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:15.349610 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d772e-3a62-4ae6-9164-7e85e289df3f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:15.435745 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714186a-cc00-47e3-8a85-bddddce6b17e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:15.487979 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:15.506140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3305f15-6bff-4abf-a339-0de88b0d38bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:15.507037 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3305f15-6bff-4abf-a339-0de88b0d38bd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:15.507316 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3305f15-6bff-4abf-a339-0de88b0d38bd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:15.507589 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3305f15-6bff-4abf-a339-0de88b0d38bd None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:15.511450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3305f15-6bff-4abf-a339-0de88b0d38bd None admin] RBAC: Authorizing `identity:get_project(project_id=e7099ea37d874a7aa73b15453887dffc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:15.512987 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3305f15-6bff-4abf-a339-0de88b0d38bd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:15.515132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 197/373] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:03:15 2022] GET /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:15.520353 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3305f15-6bff-4abf-a339-0de88b0d38bd None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:15.562030 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d772e-3a62-4ae6-9164-7e85e289df3f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:15.618227 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:15.630637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def9388-371c-41e7-b256-b89543b90e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:15.631327 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def9388-371c-41e7-b256-b89543b90e13 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:15.631625 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def9388-371c-41e7-b256-b89543b90e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:15.631930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def9388-371c-41e7-b256-b89543b90e13 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:15.632690 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def9388-371c-41e7-b256-b89543b90e13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:15.633000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def9388-371c-41e7-b256-b89543b90e13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:15.634534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def9388-371c-41e7-b256-b89543b90e13 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:15.645853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 177/374] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:15.657315 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def9388-371c-41e7-b256-b89543b90e13 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:15.735899 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3305f15-6bff-4abf-a339-0de88b0d38bd None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:15.812241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038c1ef-d828-4b62-9ce3-2599ca10d4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:15.813140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038c1ef-d828-4b62-9ce3-2599ca10d4be None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:15.813416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038c1ef-d828-4b62-9ce3-2599ca10d4be None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:15.813687 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038c1ef-d828-4b62-9ce3-2599ca10d4be None admin] PATH_INFO: `/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:15.816150 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038c1ef-d828-4b62-9ce3-2599ca10d4be None admin] RBAC: Authorizing `identity:get_group(group_id=04f5c67e865c4a30a1bd97093e65cbcd)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:15.817655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038c1ef-d828-4b62-9ce3-2599ca10d4be None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:15.819440 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 198/375] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Jun 27 21:03:15 2022] GET /identity/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd => generated 226 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:15.827146 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038c1ef-d828-4b62-9ce3-2599ca10d4be None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:15.853786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def9388-371c-41e7-b256-b89543b90e13 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:15.936725 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:15.952353 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565fbae-5fa7-4cfd-a298-1233782219c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:15.953044 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565fbae-5fa7-4cfd-a298-1233782219c0 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:15.953328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565fbae-5fa7-4cfd-a298-1233782219c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:15.953612 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565fbae-5fa7-4cfd-a298-1233782219c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:15.974839 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038c1ef-d828-4b62-9ce3-2599ca10d4be None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:16.012961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35d5e5-e2e8-4d74-b2d4-3d72eaffa0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:16.013870 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35d5e5-e2e8-4d74-b2d4-3d72eaffa0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:16.014313 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35d5e5-e2e8-4d74-b2d4-3d72eaffa0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:16.014723 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35d5e5-e2e8-4d74-b2d4-3d72eaffa0f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:16.015732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35d5e5-e2e8-4d74-b2d4-3d72eaffa0f5 None admin] RBAC: Adding query filter params (group.id=04f5c67e865c4a30a1bd97093e65cbcd, role.id=b3efb2b2e43c4af8b1cf36a37f1e05f9, scope.project.id=e7099ea37d874a7aa73b15453887dffc) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:16.016194 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35d5e5-e2e8-4d74-b2d4-3d72eaffa0f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:16.017922 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35d5e5-e2e8-4d74-b2d4-3d72eaffa0f5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:16.024504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 199/376] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Jun 27 21:03:15 2022] GET /identity/v3/role_assignments?group.id=04f5c67e865c4a30a1bd97093e65cbcd&scope.project.id=e7099ea37d874a7aa73b15453887dffc&role.id=b3efb2b2e43c4af8b1cf36a37f1e05f9 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:16.025791 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59c72c-a79d-450d-aed3-a67c70d5d72d None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:16.026078 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59c72c-a79d-450d-aed3-a67c70d5d72d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:16.026352 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59c72c-a79d-450d-aed3-a67c70d5d72d None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:16.027935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 200/377] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:16.034427 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c3e36-545c-4ac7-b432-7158f296b227 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:16.034765 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c3e36-545c-4ac7-b432-7158f296b227 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:16.034986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c3e36-545c-4ac7-b432-7158f296b227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:16.053088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0a2c3e36-545c-4ac7-b432-7158f296b227 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:16.055797 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0a2c3e36-545c-4ac7-b432-7158f296b227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:16.068799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:16.086909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565fbae-5fa7-4cfd-a298-1233782219c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:16.089408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565fbae-5fa7-4cfd-a298-1233782219c0 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:16.141888 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 201/378] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:03:16.167119 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c3e36-545c-4ac7-b432-7158f296b227 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:16.237100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 178/379] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 21:03:16.242989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565fbae-5fa7-4cfd-a298-1233782219c0 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:16.268098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:16.333326 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c3e36-545c-4ac7-b432-7158f296b227 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:16.370464 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915eab8-3ba8-4171-a4aa-1aa7ea32faee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:16.371270 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915eab8-3ba8-4171-a4aa-1aa7ea32faee None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:16.371547 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915eab8-3ba8-4171-a4aa-1aa7ea32faee None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:16.371849 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915eab8-3ba8-4171-a4aa-1aa7ea32faee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:16.374453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:16.382576 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e915eab8-3ba8-4171-a4aa-1aa7ea32faee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:16.388300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565fbae-5fa7-4cfd-a298-1233782219c0 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:16.390467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-e915eab8-3ba8-4171-a4aa-1aa7ea32faee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:03:16.390467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:16.390467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:16.390467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:16.390467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:16.390467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:16.390467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:16.390467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:16.390467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:16.390467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:16.390467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:16.390467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:16.390467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:16.390467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:16.390467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:16.390467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:16.391402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:16.391402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:16.391402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:16.391402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:16.391402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:16.391402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:16.391402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:16.391402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:16.391402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:16.391402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:16.391402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:16.391402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:16.391402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:16.391402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:16.391402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:16.391402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:16.392211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:16.392211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:16.392211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:16.392211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:16.392211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:16.392211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:16.392211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:16.392211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:16.392211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:16.392211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:16.392211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:03:16.392211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:16.395953 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 202/380] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:16.399057 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915eab8-3ba8-4171-a4aa-1aa7ea32faee None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:16.429462 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85588ac2-a25e-4259-bfe7-8f46abe60223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:16.430813 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85588ac2-a25e-4259-bfe7-8f46abe60223 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:16.431332 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85588ac2-a25e-4259-bfe7-8f46abe60223 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:16.431897 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85588ac2-a25e-4259-bfe7-8f46abe60223 None admin] PATH_INFO: `/v3/projects/2213409bed61490ba55498fcd7bd10f8` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:16.433302 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85588ac2-a25e-4259-bfe7-8f46abe60223 None admin] RBAC: Authorizing `identity:get_project(project_id=2213409bed61490ba55498fcd7bd10f8)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:16.435073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85588ac2-a25e-4259-bfe7-8f46abe60223 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:16.437361 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 179/381] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:03:16 2022] GET /identity/v3/projects/2213409bed61490ba55498fcd7bd10f8 => generated 344 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:16.444712 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85588ac2-a25e-4259-bfe7-8f46abe60223 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:16.510567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915eab8-3ba8-4171-a4aa-1aa7ea32faee None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:16.527191 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:16.547385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f296fb-1fae-4785-81e3-b644b134c8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:16.548105 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f296fb-1fae-4785-81e3-b644b134c8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:16.548385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f296fb-1fae-4785-81e3-b644b134c8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:16.548660 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f296fb-1fae-4785-81e3-b644b134c8a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:16.549472 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f296fb-1fae-4785-81e3-b644b134c8a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:16.549779 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f296fb-1fae-4785-81e3-b644b134c8a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:16.551204 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f296fb-1fae-4785-81e3-b644b134c8a3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:16.560331 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 203/382] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:03:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:16.564494 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85588ac2-a25e-4259-bfe7-8f46abe60223 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:16.565776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f296fb-1fae-4785-81e3-b644b134c8a3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:16.603076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ca01b-8bc2-45f4-a6f5-8ccf65c2852a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:16.604461 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ca01b-8bc2-45f4-a6f5-8ccf65c2852a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:16.605036 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ca01b-8bc2-45f4-a6f5-8ccf65c2852a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:16.605509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ca01b-8bc2-45f4-a6f5-8ccf65c2852a None admin] PATH_INFO: `/v3/groups/56c3d159c98640d180177ccc434665f5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:16.612542 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ca01b-8bc2-45f4-a6f5-8ccf65c2852a None admin] RBAC: Authorizing `identity:get_group(group_id=56c3d159c98640d180177ccc434665f5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:16.614396 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ca01b-8bc2-45f4-a6f5-8ccf65c2852a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:16.616785 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 180/383] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Jun 27 21:03:16 2022] GET /identity/v3/groups/56c3d159c98640d180177ccc434665f5 => generated 229 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:16.623091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ca01b-8bc2-45f4-a6f5-8ccf65c2852a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:16.658256 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:16.688289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f296fb-1fae-4785-81e3-b644b134c8a3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:16.727791 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1148b43-0cd5-4b60-b9e6-51ee4e39ccdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:16.728669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1148b43-0cd5-4b60-b9e6-51ee4e39ccdb None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:16.728946 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1148b43-0cd5-4b60-b9e6-51ee4e39ccdb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:16.729219 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1148b43-0cd5-4b60-b9e6-51ee4e39ccdb None admin] PATH_INFO: `/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:16.730455 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1148b43-0cd5-4b60-b9e6-51ee4e39ccdb None admin] RBAC: Authorizing `identity:get_group(group_id=04f5c67e865c4a30a1bd97093e65cbcd)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:16.732106 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1148b43-0cd5-4b60-b9e6-51ee4e39ccdb None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:16.734158 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 204/384] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Jun 27 21:03:16 2022] GET /identity/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:16.739785 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1148b43-0cd5-4b60-b9e6-51ee4e39ccdb None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:16.765022 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ca01b-8bc2-45f4-a6f5-8ccf65c2852a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:16.795653 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:16.811176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2fdd6-445d-420d-8abd-7bbf53b0c927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:16.812349 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2fdd6-445d-420d-8abd-7bbf53b0c927 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:16.813149 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2fdd6-445d-420d-8abd-7bbf53b0c927 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:16.813643 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2fdd6-445d-420d-8abd-7bbf53b0c927 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:16.814758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2fdd6-445d-420d-8abd-7bbf53b0c927 None admin] RBAC: Adding query filter params (group.id=56c3d159c98640d180177ccc434665f5, role.id=5c3c095bb53943eca95c4e049cd8cbf5, scope.project.id=2213409bed61490ba55498fcd7bd10f8) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:16.815528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2fdd6-445d-420d-8abd-7bbf53b0c927 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:16.817229 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2fdd6-445d-420d-8abd-7bbf53b0c927 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:16.825542 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 181/385] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Jun 27 21:03:16 2022] GET /identity/v3/role_assignments?group.id=56c3d159c98640d180177ccc434665f5&scope.project.id=2213409bed61490ba55498fcd7bd10f8&role.id=5c3c095bb53943eca95c4e049cd8cbf5 => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:16.890606 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1148b43-0cd5-4b60-b9e6-51ee4e39ccdb None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:16.942398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742b0a2-17cc-493d-8d39-ab68b89640f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:16.943291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742b0a2-17cc-493d-8d39-ab68b89640f6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:16.943567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742b0a2-17cc-493d-8d39-ab68b89640f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:16.943864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742b0a2-17cc-493d-8d39-ab68b89640f6 None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:16.945042 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742b0a2-17cc-493d-8d39-ab68b89640f6 None admin] RBAC: Authorizing `identity:get_project(project_id=0dc9df6fada54f72a35899fa21f03b20)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:16.946506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742b0a2-17cc-493d-8d39-ab68b89640f6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:16.948534 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 205/386] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:03:16 2022] GET /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:16.954315 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2fdd6-445d-420d-8abd-7bbf53b0c927 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:17.052898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a55c6-c370-426c-87a9-d71524fba811 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:17.053082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a55c6-c370-426c-87a9-d71524fba811 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:17.053358 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a55c6-c370-426c-87a9-d71524fba811 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:17.054741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 206/387] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:17.060304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17451f2-6bbb-4e65-a837-df5aa0935cdb None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:17.060582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17451f2-6bbb-4e65-a837-df5aa0935cdb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:17.060855 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17451f2-6bbb-4e65-a837-df5aa0935cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:17.086782 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:17.110992 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2fdd6-445d-420d-8abd-7bbf53b0c927 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:17.111844 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e17451f2-6bbb-4e65-a837-df5aa0935cdb None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:17.113170 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e17451f2-6bbb-4e65-a837-df5aa0935cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:17.174196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d730d42-e0d1-458e-8f29-3c09c2d834ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:17.176324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d730d42-e0d1-458e-8f29-3c09c2d834ae None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:17.176657 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d730d42-e0d1-458e-8f29-3c09c2d834ae None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:17.176936 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d730d42-e0d1-458e-8f29-3c09c2d834ae None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20/groups/04f5c67e865c4a30a1bd97093e65cbcd/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:17.181178 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d730d42-e0d1-458e-8f29-3c09c2d834ae None admin] RBAC: Authorizing `identity:create_grant(project_id=0dc9df6fada54f72a35899fa21f03b20, group_id=04f5c67e865c4a30a1bd97093e65cbcd, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:17.184436 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d730d42-e0d1-458e-8f29-3c09c2d834ae None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:17.209865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 182/388] 10.222.0.27 () {58 vars in 1442 bytes} [Mon Jun 27 21:03:16 2022] PUT /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20/groups/04f5c67e865c4a30a1bd97093e65cbcd/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:17.231470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 207/389] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 27 21:03:17.264607 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d730d42-e0d1-458e-8f29-3c09c2d834ae None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:17.400505 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d730d42-e0d1-458e-8f29-3c09c2d834ae None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:17.439067 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4782bdc-3c9d-49f4-91a3-02a832487b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:17.440036 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4782bdc-3c9d-49f4-91a3-02a832487b1e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:17.440333 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4782bdc-3c9d-49f4-91a3-02a832487b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:17.440588 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4782bdc-3c9d-49f4-91a3-02a832487b1e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:17.451345 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4782bdc-3c9d-49f4-91a3-02a832487b1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:17.459723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-b4782bdc-3c9d-49f4-91a3-02a832487b1e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:03:17.459723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:17.459723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:17.459723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:17.459723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:17.459723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:17.459723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:17.459723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:17.459723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:17.459723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:17.459723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:17.459723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:17.459723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:17.459723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:17.459723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:17.459723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:17.460632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:17.460632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:17.460632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:17.460632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:17.460632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:17.460632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:17.460632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:17.460632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:17.460632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:17.460632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:17.460632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:17.460632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:17.460632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:17.460632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:17.460632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:17.460632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:17.461400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:17.461400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:17.461400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:17.461400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:17.461400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:17.461400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:17.461400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:17.461400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:17.461400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:17.461400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:17.461400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:03:17.461400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:17.461400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 183/390] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:03:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:17.467091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17451f2-6bbb-4e65-a837-df5aa0935cdb None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:17.519917 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:17.581093 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17451f2-6bbb-4e65-a837-df5aa0935cdb None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:17.617854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110ec86-5f81-4613-9c9b-4b3958db76bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:17.618533 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110ec86-5f81-4613-9c9b-4b3958db76bd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:17.618814 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110ec86-5f81-4613-9c9b-4b3958db76bd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:17.619081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110ec86-5f81-4613-9c9b-4b3958db76bd None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:17.619866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110ec86-5f81-4613-9c9b-4b3958db76bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:17.620173 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110ec86-5f81-4613-9c9b-4b3958db76bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:17.621553 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110ec86-5f81-4613-9c9b-4b3958db76bd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:17.630068 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 208/391] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:17.636014 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4782bdc-3c9d-49f4-91a3-02a832487b1e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:17.640551 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:17.751376 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af1e45-4019-48ad-822e-d332df94c441 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:17.751890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af1e45-4019-48ad-822e-d332df94c441 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:17.751890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af1e45-4019-48ad-822e-d332df94c441 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:17.754622 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 209/392] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:17.760124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c33b02-36a5-467c-93e3-7b6a5bfaee4f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:17.760406 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c33b02-36a5-467c-93e3-7b6a5bfaee4f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:17.760681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c33b02-36a5-467c-93e3-7b6a5bfaee4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:17.789514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-72c33b02-36a5-467c-93e3-7b6a5bfaee4f None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:17.790707 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-72c33b02-36a5-467c-93e3-7b6a5bfaee4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:17.821498 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4782bdc-3c9d-49f4-91a3-02a832487b1e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:17.874903 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca0331-ae81-404b-98ee-5d5eb340f36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:17.876234 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca0331-ae81-404b-98ee-5d5eb340f36e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:17.876721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca0331-ae81-404b-98ee-5d5eb340f36e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:17.877234 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca0331-ae81-404b-98ee-5d5eb340f36e None admin] PATH_INFO: `/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:17.880659 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca0331-ae81-404b-98ee-5d5eb340f36e None admin] RBAC: Authorizing `identity:get_group(group_id=04f5c67e865c4a30a1bd97093e65cbcd)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:17.882493 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca0331-ae81-404b-98ee-5d5eb340f36e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:17.884535 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 184/393] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Jun 27 21:03:17 2022] GET /identity/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:17.890125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca0331-ae81-404b-98ee-5d5eb340f36e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:17.905819 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 210/394] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:17.928784 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c33b02-36a5-467c-93e3-7b6a5bfaee4f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:17.964748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:18.010810 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:18.032377 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c33b02-36a5-467c-93e3-7b6a5bfaee4f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:18.053681 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca0331-ae81-404b-98ee-5d5eb340f36e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:18.069532 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404082ef-ce9b-4485-808c-6b2e4b679a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:18.070363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404082ef-ce9b-4485-808c-6b2e4b679a08 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:18.070660 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404082ef-ce9b-4485-808c-6b2e4b679a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:18.070936 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404082ef-ce9b-4485-808c-6b2e4b679a08 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:18.082149 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404082ef-ce9b-4485-808c-6b2e4b679a08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:18.094794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-404082ef-ce9b-4485-808c-6b2e4b679a08 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:03:18.094794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:18.094794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:18.094794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:18.094794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:18.094794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:18.094794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:18.094794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:18.094794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:18.094794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:18.094794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:18.094794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:18.094794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:18.094794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:18.094794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:18.094794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:18.095696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:18.095696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:18.095696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:18.095696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:18.095696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:18.095696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:18.095696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:18.095696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:18.095696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:18.095696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:18.095696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:18.095696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:18.095696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:18.095696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:18.095696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:18.095696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:18.096609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:18.096609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:18.096609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:18.096609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:18.096609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:18.096609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:18.096609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:18.096609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:18.096609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:18.096609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:18.096609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:03:18.096609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:18.099194 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 211/395] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:18.107728 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404082ef-ce9b-4485-808c-6b2e4b679a08 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:18.121032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cacd62-895f-4481-90cd-6907a9a80c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:18.121927 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cacd62-895f-4481-90cd-6907a9a80c2b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:18.122205 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cacd62-895f-4481-90cd-6907a9a80c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:18.122481 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cacd62-895f-4481-90cd-6907a9a80c2b None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:18.125729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cacd62-895f-4481-90cd-6907a9a80c2b None admin] RBAC: Authorizing `identity:get_project(project_id=0dc9df6fada54f72a35899fa21f03b20)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:18.127317 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cacd62-895f-4481-90cd-6907a9a80c2b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:18.130841 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 185/396] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:03:17 2022] GET /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:18.134734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cacd62-895f-4481-90cd-6907a9a80c2b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:18.283273 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:18.291344 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404082ef-ce9b-4485-808c-6b2e4b679a08 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:18.301759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cacd62-895f-4481-90cd-6907a9a80c2b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:18.326514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f742c-f46c-4419-b449-da061754849a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:18.327210 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f742c-f46c-4419-b449-da061754849a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:18.327495 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f742c-f46c-4419-b449-da061754849a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:18.327806 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f742c-f46c-4419-b449-da061754849a None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:18.328571 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f742c-f46c-4419-b449-da061754849a None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:18.328883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f742c-f46c-4419-b449-da061754849a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:18.330332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f742c-f46c-4419-b449-da061754849a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:18.339524 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 212/397] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:03:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:18.345152 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f742c-f46c-4419-b449-da061754849a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:18.417276 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bcd6d-0a86-438b-82e1-5bc439df239b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:18.418191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bcd6d-0a86-438b-82e1-5bc439df239b None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:18.418473 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bcd6d-0a86-438b-82e1-5bc439df239b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:18.418749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bcd6d-0a86-438b-82e1-5bc439df239b None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20/groups/04f5c67e865c4a30a1bd97093e65cbcd/roles/b3efb2b2e43c4af8b1cf36a37f1e05f9` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:18.432472 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091bcd6d-0a86-438b-82e1-5bc439df239b None admin] RBAC: Authorizing `identity:create_grant(project_id=0dc9df6fada54f72a35899fa21f03b20, group_id=04f5c67e865c4a30a1bd97093e65cbcd, role_id=b3efb2b2e43c4af8b1cf36a37f1e05f9)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:18.433949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091bcd6d-0a86-438b-82e1-5bc439df239b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:18.448714 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f742c-f46c-4419-b449-da061754849a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:18.462845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:18.477729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 186/398] 10.222.0.27 () {58 vars in 1442 bytes} [Mon Jun 27 21:03:18 2022] PUT /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20/groups/04f5c67e865c4a30a1bd97093e65cbcd/roles/b3efb2b2e43c4af8b1cf36a37f1e05f9 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:03:18.487428 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44cc31-58fa-4c8a-b6e5-7afe722e50ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:18.488445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44cc31-58fa-4c8a-b6e5-7afe722e50ba None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:18.488908 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44cc31-58fa-4c8a-b6e5-7afe722e50ba None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:18.489289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44cc31-58fa-4c8a-b6e5-7afe722e50ba None admin] PATH_INFO: `/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:18.490570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44cc31-58fa-4c8a-b6e5-7afe722e50ba None admin] RBAC: Authorizing `identity:get_group(group_id=04f5c67e865c4a30a1bd97093e65cbcd)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:18.492242 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44cc31-58fa-4c8a-b6e5-7afe722e50ba None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:18.494215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 213/399] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Jun 27 21:03:18 2022] GET /identity/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:18.499698 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bcd6d-0a86-438b-82e1-5bc439df239b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:18.611503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bcd6d-0a86-438b-82e1-5bc439df239b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:18.647351 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d70180-af80-4533-ad64-d87a25969fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:18.648272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d70180-af80-4533-ad64-d87a25969fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:18.648551 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d70180-af80-4533-ad64-d87a25969fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:18.648824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d70180-af80-4533-ad64-d87a25969fb6 None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:18.650052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d70180-af80-4533-ad64-d87a25969fb6 None admin] RBAC: Authorizing `identity:get_project(project_id=e7099ea37d874a7aa73b15453887dffc)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:18.651530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d70180-af80-4533-ad64-d87a25969fb6 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:18.653611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 187/400] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:03:18 2022] GET /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:18.659907 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44cc31-58fa-4c8a-b6e5-7afe722e50ba None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:18.690864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:18.768662 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44cc31-58fa-4c8a-b6e5-7afe722e50ba None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:18.814263 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c8fd5-07af-461c-bc4f-8a92fd0931c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:18.815667 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c8fd5-07af-461c-bc4f-8a92fd0931c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:18.816144 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c8fd5-07af-461c-bc4f-8a92fd0931c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:18.816624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c8fd5-07af-461c-bc4f-8a92fd0931c5 None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc/groups/04f5c67e865c4a30a1bd97093e65cbcd/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:18.819544 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c8fd5-07af-461c-bc4f-8a92fd0931c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=e7099ea37d874a7aa73b15453887dffc, group_id=04f5c67e865c4a30a1bd97093e65cbcd, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:18.821264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c8fd5-07af-461c-bc4f-8a92fd0931c5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:18.841387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 214/401] 10.222.0.27 () {58 vars in 1442 bytes} [Mon Jun 27 21:03:18 2022] PUT /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc/groups/04f5c67e865c4a30a1bd97093e65cbcd/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:03:19.124931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb21ac-1aa4-408f-af9b-b7d4dec146f4 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:19.125926 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb21ac-1aa4-408f-af9b-b7d4dec146f4 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:19.126408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb21ac-1aa4-408f-af9b-b7d4dec146f4 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:19.128392 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 188/402] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:19.133618 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b16b1-9719-4c95-8acf-d02e4c4ebfe1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:19.134154 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b16b1-9719-4c95-8acf-d02e4c4ebfe1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:19.134626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b16b1-9719-4c95-8acf-d02e4c4ebfe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:19.155898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4d7b16b1-9719-4c95-8acf-d02e4c4ebfe1 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:19.157434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4d7b16b1-9719-4c95-8acf-d02e4c4ebfe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:19.242803 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 215/403] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:19.276970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb21ac-1aa4-408f-af9b-b7d4dec146f4 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:19.367638 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:19.410485 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb21ac-1aa4-408f-af9b-b7d4dec146f4 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:19.444887 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b0b08-39be-40b8-8d95-52359f9ea761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:19.445714 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b0b08-39be-40b8-8d95-52359f9ea761 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:19.445991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b0b08-39be-40b8-8d95-52359f9ea761 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:19.446264 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b0b08-39be-40b8-8d95-52359f9ea761 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:19.457147 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b0b08-39be-40b8-8d95-52359f9ea761 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:19.465376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-595b0b08-39be-40b8-8d95-52359f9ea761 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:03:19.465376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:19.465376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:19.465376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:19.465376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:19.465376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:19.465376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:19.465376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:19.465376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:19.465376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:19.465376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:19.465376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:19.465376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:19.465376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:19.465376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:19.465376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:19.466250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:19.466250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:19.466250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:19.466250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:19.466250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:19.466250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:19.466250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:19.466250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:19.466250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:19.466250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:19.466250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:19.466250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:19.466250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:19.466250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:19.466250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:19.466250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:19.467015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:19.467015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:19.467015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:19.467015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:19.467015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:19.467015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:19.467015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:19.467015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:19.467015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:19.467015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:19.467015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:03:19.467015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:19.467015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 189/404] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:03:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:19.473101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b16b1-9719-4c95-8acf-d02e4c4ebfe1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:19.500227 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:19.593870 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b16b1-9719-4c95-8acf-d02e4c4ebfe1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:19.634695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb51a42-c855-4bb2-8611-0bce91f6ebe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:19.635835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb51a42-c855-4bb2-8611-0bce91f6ebe1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:19.636329 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb51a42-c855-4bb2-8611-0bce91f6ebe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:19.636795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb51a42-c855-4bb2-8611-0bce91f6ebe1 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:19.637746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb51a42-c855-4bb2-8611-0bce91f6ebe1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:19.638243 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb51a42-c855-4bb2-8611-0bce91f6ebe1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:19.639863 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb51a42-c855-4bb2-8611-0bce91f6ebe1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:19.649456 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 216/405] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:19.654836 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b0b08-39be-40b8-8d95-52359f9ea761 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:19.770463 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b0b08-39be-40b8-8d95-52359f9ea761 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:19.797640 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:19.799928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a6458-939c-4956-9b90-1b0ffdc5004f None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:19.801169 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a6458-939c-4956-9b90-1b0ffdc5004f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:19.802210 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a6458-939c-4956-9b90-1b0ffdc5004f None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:19.809356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 217/406] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:19 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:19.813375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bde351-f178-4f25-ba10-e6018255c639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:19.814271 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bde351-f178-4f25-ba10-e6018255c639 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:19.814612 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bde351-f178-4f25-ba10-e6018255c639 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:19.814892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bde351-f178-4f25-ba10-e6018255c639 None admin] PATH_INFO: `/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:19.816125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bde351-f178-4f25-ba10-e6018255c639 None admin] RBAC: Authorizing `identity:get_group(group_id=04f5c67e865c4a30a1bd97093e65cbcd)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:19.816937 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73415d25-8a6c-40ca-8609-bff564bf6d56 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:19.817223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73415d25-8a6c-40ca-8609-bff564bf6d56 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:19.817501 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73415d25-8a6c-40ca-8609-bff564bf6d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:19.817611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bde351-f178-4f25-ba10-e6018255c639 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:19.819496 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 190/407] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Jun 27 21:03:19 2022] GET /identity/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:19.824582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bde351-f178-4f25-ba10-e6018255c639 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:19.849491 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-73415d25-8a6c-40ca-8609-bff564bf6d56 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:19.850694 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-73415d25-8a6c-40ca-8609-bff564bf6d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:19.937262 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 218/408] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:19.939287 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f752e-b028-492f-bbe7-b6ba404ac506 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:19.939933 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f752e-b028-492f-bbe7-b6ba404ac506 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:19.940425 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f752e-b028-492f-bbe7-b6ba404ac506 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:19.941884 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 219/409] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:19.947385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d421d64-aa63-4ac7-a134-4dab1bcf70ee None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:19.948171 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d421d64-aa63-4ac7-a134-4dab1bcf70ee None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:19.948648 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d421d64-aa63-4ac7-a134-4dab1bcf70ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:19.968011 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8d421d64-aa63-4ac7-a134-4dab1bcf70ee None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:19.969758 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8d421d64-aa63-4ac7-a134-4dab1bcf70ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:19.972346 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:19.987154 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bde351-f178-4f25-ba10-e6018255c639 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:20.028736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ce549-267e-490b-ab19-8b113dc1af77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:20.029976 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ce549-267e-490b-ab19-8b113dc1af77 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:20.030429 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ce549-267e-490b-ab19-8b113dc1af77 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:20.030897 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ce549-267e-490b-ab19-8b113dc1af77 None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:20.032375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ce549-267e-490b-ab19-8b113dc1af77 None admin] RBAC: Authorizing `identity:get_project(project_id=e7099ea37d874a7aa73b15453887dffc)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:20.034461 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ce549-267e-490b-ab19-8b113dc1af77 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:20.036751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 191/410] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:03:19 2022] GET /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:20.037831 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ce549-267e-490b-ab19-8b113dc1af77 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:20.067693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 220/411] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:20.068860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d421d64-aa63-4ac7-a134-4dab1bcf70ee None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:20.129273 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:20.180972 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ce549-267e-490b-ab19-8b113dc1af77 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:20.195572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d421d64-aa63-4ac7-a134-4dab1bcf70ee None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:20.225845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801bf2b-4c9f-49c3-967f-5d583a139fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:20.226981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801bf2b-4c9f-49c3-967f-5d583a139fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:20.227360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801bf2b-4c9f-49c3-967f-5d583a139fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:20.227824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801bf2b-4c9f-49c3-967f-5d583a139fc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:20.233149 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e302c80-f547-481e-9290-8bac9d86896a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:20.234263 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e302c80-f547-481e-9290-8bac9d86896a None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:20.234716 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e302c80-f547-481e-9290-8bac9d86896a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:20.235176 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e302c80-f547-481e-9290-8bac9d86896a None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc/groups/04f5c67e865c4a30a1bd97093e65cbcd/roles/b3efb2b2e43c4af8b1cf36a37f1e05f9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:20.237847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e302c80-f547-481e-9290-8bac9d86896a None admin] RBAC: Authorizing `identity:create_grant(project_id=e7099ea37d874a7aa73b15453887dffc, group_id=04f5c67e865c4a30a1bd97093e65cbcd, role_id=b3efb2b2e43c4af8b1cf36a37f1e05f9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:20.239475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e302c80-f547-481e-9290-8bac9d86896a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:20.239598 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801bf2b-4c9f-49c3-967f-5d583a139fc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:20.249741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-5801bf2b-4c9f-49c3-967f-5d583a139fc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:20.249741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:20.249741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:20.249741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:20.249741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:20.249741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:20.249741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:20.249741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:20.249741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:20.249741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:20.249741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:20.249741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:20.249741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:20.249741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:20.249741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:20.249741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:20.250687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:20.250687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:20.250687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:20.250687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:20.250687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:20.250687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:20.250687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:20.250687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:20.250687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:20.250687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:20.250687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:20.250687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:20.250687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:20.250687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:20.250687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:20.250687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:20.251491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:20.251491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:20.251491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:20.251491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:20.251491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:20.251491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:20.251491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:20.251491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:20.251491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:20.251491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:20.251491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:20.251491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:20.251491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 192/412] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:20.253363 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801bf2b-4c9f-49c3-967f-5d583a139fc7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:20.259351 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:20.270352 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 221/413] 10.222.0.27 () {58 vars in 1442 bytes} [Mon Jun 27 21:03:20 2022] PUT /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc/groups/04f5c67e865c4a30a1bd97093e65cbcd/roles/b3efb2b2e43c4af8b1cf36a37f1e05f9 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:20.271454 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e302c80-f547-481e-9290-8bac9d86896a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:20.389691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e302c80-f547-481e-9290-8bac9d86896a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:20.397793 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801bf2b-4c9f-49c3-967f-5d583a139fc7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:20.423436 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:20.445340 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b68929-2c50-4d91-a6f0-e5624332557a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:20.446065 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b68929-2c50-4d91-a6f0-e5624332557a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:20.446440 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b68929-2c50-4d91-a6f0-e5624332557a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:20.446728 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b68929-2c50-4d91-a6f0-e5624332557a None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:20.447592 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b68929-2c50-4d91-a6f0-e5624332557a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:20.448008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b68929-2c50-4d91-a6f0-e5624332557a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:20.449488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b68929-2c50-4d91-a6f0-e5624332557a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:20.453147 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da7b86-d174-4dc3-bfa8-8febdadad2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:20.454236 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da7b86-d174-4dc3-bfa8-8febdadad2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:20.454734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da7b86-d174-4dc3-bfa8-8febdadad2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:20.455259 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da7b86-d174-4dc3-bfa8-8febdadad2e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:20.459619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 222/414] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:20.465638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b68929-2c50-4d91-a6f0-e5624332557a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:20.481328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da7b86-d174-4dc3-bfa8-8febdadad2e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:20.500189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-83da7b86-d174-4dc3-bfa8-8febdadad2e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:03:20.500189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:20.500189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:20.500189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:20.500189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:20.500189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:20.500189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:20.500189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:20.500189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:20.500189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:20.500189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:20.500189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:20.500189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:20.500189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:20.500189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:20.500189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:20.501129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:20.501129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:20.501129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:20.501129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:20.501129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:20.501129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:20.501129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:20.501129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:20.501129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:20.501129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:20.501129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:20.501129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:20.501129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:20.501129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:20.501129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:20.501129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:20.501911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:20.501911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:20.501911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:20.501911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:20.501911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:20.501911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:20.501911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:20.501911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:20.501911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:20.501911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:20.501911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:03:20.501911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:20.504465 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 193/415] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:20.510495 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da7b86-d174-4dc3-bfa8-8febdadad2e6 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:20.535169 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:20.582772 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b68929-2c50-4d91-a6f0-e5624332557a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:20.623472 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ef34b-9318-4291-9e28-b5168cb16499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:20.624614 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ef34b-9318-4291-9e28-b5168cb16499 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:20.625055 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ef34b-9318-4291-9e28-b5168cb16499 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:20.625529 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ef34b-9318-4291-9e28-b5168cb16499 None admin] PATH_INFO: `/v3/groups/56c3d159c98640d180177ccc434665f5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:20.627351 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ef34b-9318-4291-9e28-b5168cb16499 None admin] RBAC: Authorizing `identity:get_group(group_id=56c3d159c98640d180177ccc434665f5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:20.629220 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ef34b-9318-4291-9e28-b5168cb16499 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:20.631418 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 223/416] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Jun 27 21:03:20 2022] GET /identity/v3/groups/56c3d159c98640d180177ccc434665f5 => generated 229 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:20.637234 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ef34b-9318-4291-9e28-b5168cb16499 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:20.646398 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da7b86-d174-4dc3-bfa8-8febdadad2e6 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:20.691213 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096bbf9d-fcc9-49d4-a5c0-1b3b17719989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:20.692794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096bbf9d-fcc9-49d4-a5c0-1b3b17719989 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:20.693075 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096bbf9d-fcc9-49d4-a5c0-1b3b17719989 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:20.693347 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096bbf9d-fcc9-49d4-a5c0-1b3b17719989 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:20.694166 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096bbf9d-fcc9-49d4-a5c0-1b3b17719989 None admin] RBAC: Adding query filter params (name=member) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:20.694477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096bbf9d-fcc9-49d4-a5c0-1b3b17719989 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:20.696583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096bbf9d-fcc9-49d4-a5c0-1b3b17719989 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:20.707061 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 194/417] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:03:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:20.715679 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096bbf9d-fcc9-49d4-a5c0-1b3b17719989 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:20.768333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ef34b-9318-4291-9e28-b5168cb16499 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:20.793651 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:20.816999 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:20.834292 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea89e23-ba9e-404d-baff-d0073bce6cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:20.835197 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea89e23-ba9e-404d-baff-d0073bce6cac None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:20.835473 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea89e23-ba9e-404d-baff-d0073bce6cac None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:20.835771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea89e23-ba9e-404d-baff-d0073bce6cac None admin] PATH_INFO: `/v3/projects/2213409bed61490ba55498fcd7bd10f8` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:20.836969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea89e23-ba9e-404d-baff-d0073bce6cac None admin] RBAC: Authorizing `identity:get_project(project_id=2213409bed61490ba55498fcd7bd10f8)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:20.838444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea89e23-ba9e-404d-baff-d0073bce6cac None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:20.849081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 224/418] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:03:20 2022] GET /identity/v3/projects/2213409bed61490ba55498fcd7bd10f8 => generated 344 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:20.854998 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea89e23-ba9e-404d-baff-d0073bce6cac None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:20.873626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096bbf9d-fcc9-49d4-a5c0-1b3b17719989 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:20.916012 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c37b6-2f59-4826-97cb-fce561c1859c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:20.916960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c37b6-2f59-4826-97cb-fce561c1859c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:20.917433 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c37b6-2f59-4826-97cb-fce561c1859c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:20.917998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c37b6-2f59-4826-97cb-fce561c1859c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.022427 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea89e23-ba9e-404d-baff-d0073bce6cac None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:21.052508 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c37b6-2f59-4826-97cb-fce561c1859c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:21.054124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c37b6-2f59-4826-97cb-fce561c1859c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:21.064057 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c4378-de75-4d4c-9e53-ea1c78232959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:21.064999 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c4378-de75-4d4c-9e53-ea1c78232959 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.065279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c4378-de75-4d4c-9e53-ea1c78232959 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.065553 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c4378-de75-4d4c-9e53-ea1c78232959 None admin] PATH_INFO: `/v3/projects/2213409bed61490ba55498fcd7bd10f8/groups/56c3d159c98640d180177ccc434665f5/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.070536 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c4378-de75-4d4c-9e53-ea1c78232959 None admin] RBAC: Authorizing `identity:create_grant(project_id=2213409bed61490ba55498fcd7bd10f8, group_id=56c3d159c98640d180177ccc434665f5, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:21.072046 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c4378-de75-4d4c-9e53-ea1c78232959 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:21.094608 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 225/419] 10.222.0.27 () {58 vars in 1442 bytes} [Mon Jun 27 21:03:20 2022] PUT /identity/v3/projects/2213409bed61490ba55498fcd7bd10f8/groups/56c3d159c98640d180177ccc434665f5/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:21.124008 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:21.147641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db418464-31d9-48e4-be3c-7724d2f6d6d7 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.148000 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db418464-31d9-48e4-be3c-7724d2f6d6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.148191 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db418464-31d9-48e4-be3c-7724d2f6d6d7 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.149700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 226/420] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:21.150619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f0cd8-675d-4fde-986a-168453657086 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.150951 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f0cd8-675d-4fde-986a-168453657086 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.151235 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f0cd8-675d-4fde-986a-168453657086 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.152596 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 227/421] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:21.157473 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a93f2-2e37-4b11-ab8c-39ab7e4b4c8c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.157762 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a93f2-2e37-4b11-ab8c-39ab7e4b4c8c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.158411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a93f2-2e37-4b11-ab8c-39ab7e4b4c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.176787 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4f5a93f2-2e37-4b11-ab8c-39ab7e4b4c8c None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:21.177936 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4f5a93f2-2e37-4b11-ab8c-39ab7e4b4c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:21.190926 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 195/422] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:21.193030 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382a146-7f2c-402a-b973-e4dc9c51ae1d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.193534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382a146-7f2c-402a-b973-e4dc9c51ae1d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.193947 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382a146-7f2c-402a-b973-e4dc9c51ae1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.216052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-4382a146-7f2c-402a-b973-e4dc9c51ae1d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:21.217634 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4382a146-7f2c-402a-b973-e4dc9c51ae1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:21.242500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:21.267631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 228/423] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:21.267631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a93f2-2e37-4b11-ab8c-39ab7e4b4c8c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:21.318310 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 196/424] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:21.320352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382a146-7f2c-402a-b973-e4dc9c51ae1d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:21.408146 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a93f2-2e37-4b11-ab8c-39ab7e4b4c8c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:21.416150 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:21.436592 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382a146-7f2c-402a-b973-e4dc9c51ae1d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:21.460920 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f3a4c-5596-43aa-af50-bd344ddf0f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:21.462190 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f3a4c-5596-43aa-af50-bd344ddf0f1a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.462676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f3a4c-5596-43aa-af50-bd344ddf0f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.463140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f3a4c-5596-43aa-af50-bd344ddf0f1a None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.464605 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f3a4c-5596-43aa-af50-bd344ddf0f1a None admin] RBAC: Authorizing `identity:get_project(project_id=0dc9df6fada54f72a35899fa21f03b20)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:21.466387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f3a4c-5596-43aa-af50-bd344ddf0f1a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:21.473048 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 229/425] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:03:21 2022] GET /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:21.473402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9fb5a-3722-4f40-8d19-85e35878042b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:21.474261 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9fb5a-3722-4f40-8d19-85e35878042b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.474545 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9fb5a-3722-4f40-8d19-85e35878042b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.474731 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f3a4c-5596-43aa-af50-bd344ddf0f1a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:21.474840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9fb5a-3722-4f40-8d19-85e35878042b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.488685 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9fb5a-3722-4f40-8d19-85e35878042b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:21.499506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-38d9fb5a-3722-4f40-8d19-85e35878042b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:03:21.499506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:21.499506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:21.499506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:21.499506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:21.499506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:21.499506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:21.499506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:21.499506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:21.499506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:21.499506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:21.499506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:21.499506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:21.499506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:21.499506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:21.499506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:21.500413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:21.500413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:21.500413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:21.500413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:21.500413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:21.500413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:21.500413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:21.500413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:21.500413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:21.500413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:21.500413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:21.500413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:21.500413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:21.500413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:21.500413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:21.500413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:21.501228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:21.501228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:21.501228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:21.501228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:21.501228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:21.501228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:21.501228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:21.501228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:21.501228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:21.501228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:21.501228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:03:21.501228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:21.503701 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 197/426] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:03:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:21.504717 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9fb5a-3722-4f40-8d19-85e35878042b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:21.538844 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:21.589662 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f3a4c-5596-43aa-af50-bd344ddf0f1a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:21.629678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee8500-0e58-4456-b49d-23e307940565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:21.630878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee8500-0e58-4456-b49d-23e307940565 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.631376 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee8500-0e58-4456-b49d-23e307940565 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.631904 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee8500-0e58-4456-b49d-23e307940565 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.636683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9fb5a-3722-4f40-8d19-85e35878042b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:21.643477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee8500-0e58-4456-b49d-23e307940565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:21.651972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-ecee8500-0e58-4456-b49d-23e307940565 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:03:21.651972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:21.651972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:21.651972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:21.651972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:21.651972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:21.651972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:21.651972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:21.651972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:21.651972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:21.651972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:21.651972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:21.651972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:21.651972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:21.651972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:21.651972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:21.652846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:21.652846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:21.652846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:21.652846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:21.652846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:21.652846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:21.652846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:21.652846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:21.652846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:21.652846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:21.652846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:21.652846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:21.652846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:21.652846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:21.652846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:21.652846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:21.653619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:21.653619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:21.653619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:21.653619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:21.653619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:21.653619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:21.653619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:21.653619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:21.653619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:21.653619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:21.653619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:03:21.653619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:21.656312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 230/427] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:21.657358 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee8500-0e58-4456-b49d-23e307940565 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:21.672762 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec5714-7222-4692-80a5-2e8c61c23ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:21.673921 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec5714-7222-4692-80a5-2e8c61c23ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.674312 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec5714-7222-4692-80a5-2e8c61c23ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.674968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec5714-7222-4692-80a5-2e8c61c23ec7 None admin] PATH_INFO: `/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.676078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec5714-7222-4692-80a5-2e8c61c23ec7 None admin] RBAC: Authorizing `identity:get_group(group_id=04f5c67e865c4a30a1bd97093e65cbcd)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:21.677837 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec5714-7222-4692-80a5-2e8c61c23ec7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:21.684292 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 198/428] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Jun 27 21:03:21 2022] GET /identity/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:21.684292 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec5714-7222-4692-80a5-2e8c61c23ec7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:21.691939 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:21.777090 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee8500-0e58-4456-b49d-23e307940565 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:21.794635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:21.812289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec5714-7222-4692-80a5-2e8c61c23ec7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:21.824143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f62fb-c6f6-42e2-815f-971edb440c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:21.825783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f62fb-c6f6-42e2-815f-971edb440c91 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.826473 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f62fb-c6f6-42e2-815f-971edb440c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.826885 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f62fb-c6f6-42e2-815f-971edb440c91 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.828073 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f62fb-c6f6-42e2-815f-971edb440c91 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:21.828524 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f62fb-c6f6-42e2-815f-971edb440c91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:21.830786 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f62fb-c6f6-42e2-815f-971edb440c91 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:21.842599 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 231/429] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:21.844253 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f62fb-c6f6-42e2-815f-971edb440c91 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:21.855945 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddc5bb-6b62-4145-9bfc-4b8623812944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:21.856625 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddc5bb-6b62-4145-9bfc-4b8623812944 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.856906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddc5bb-6b62-4145-9bfc-4b8623812944 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.857183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddc5bb-6b62-4145-9bfc-4b8623812944 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.858177 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bddc5bb-6b62-4145-9bfc-4b8623812944 None admin] RBAC: Adding query filter params (name=member) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:21.858283 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bddc5bb-6b62-4145-9bfc-4b8623812944 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:21.859648 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bddc5bb-6b62-4145-9bfc-4b8623812944 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:21.868603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 199/430] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:03:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:21.869415 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddc5bb-6b62-4145-9bfc-4b8623812944 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:21.946972 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:21.980716 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f62fb-c6f6-42e2-815f-971edb440c91 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:21.998012 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddc5bb-6b62-4145-9bfc-4b8623812944 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:22.022551 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad683f-7066-440c-ac60-425293279739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:22.023212 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad683f-7066-440c-ac60-425293279739 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:22.023490 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad683f-7066-440c-ac60-425293279739 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:22.023814 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad683f-7066-440c-ac60-425293279739 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:22.024648 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad683f-7066-440c-ac60-425293279739 None admin] RBAC: Adding query filter params (group.id=04f5c67e865c4a30a1bd97093e65cbcd, role.id=b1bcb7445ee54636ae7b29a2485cc201, scope.project.id=0dc9df6fada54f72a35899fa21f03b20) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:22.024964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad683f-7066-440c-ac60-425293279739 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:22.026555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad683f-7066-440c-ac60-425293279739 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:22.033424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 232/431] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Jun 27 21:03:21 2022] GET /identity/v3/role_assignments?group.id=04f5c67e865c4a30a1bd97093e65cbcd&scope.project.id=0dc9df6fada54f72a35899fa21f03b20&role.id=b1bcb7445ee54636ae7b29a2485cc201 => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:22.034224 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad683f-7066-440c-ac60-425293279739 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:22.054886 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3e47c-da9c-4eda-a6f0-f57db97420ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:22.055573 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3e47c-da9c-4eda-a6f0-f57db97420ce None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:22.055877 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3e47c-da9c-4eda-a6f0-f57db97420ce None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:22.056150 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3e47c-da9c-4eda-a6f0-f57db97420ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:22.098076 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:22.204603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3e47c-da9c-4eda-a6f0-f57db97420ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:22.205636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3e47c-da9c-4eda-a6f0-f57db97420ce None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:22.209196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad683f-7066-440c-ac60-425293279739 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:22.249131 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0164cbc-f838-4b68-a876-f7a7d6d383b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:22.250139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0164cbc-f838-4b68-a876-f7a7d6d383b5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:22.250652 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0164cbc-f838-4b68-a876-f7a7d6d383b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:22.251146 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0164cbc-f838-4b68-a876-f7a7d6d383b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:22.287989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:22.334518 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0164cbc-f838-4b68-a876-f7a7d6d383b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:22.336534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 200/432] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:22.336651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0164cbc-f838-4b68-a876-f7a7d6d383b5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:22.342115 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3e47c-da9c-4eda-a6f0-f57db97420ce None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:22.390499 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:22.471903 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 233/433] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:22.477746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3e47c-da9c-4eda-a6f0-f57db97420ce None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:22.478902 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0164cbc-f838-4b68-a876-f7a7d6d383b5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:22.519726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635cb1b-448d-4ce6-a8e2-23d2b25963be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:22.520999 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635cb1b-448d-4ce6-a8e2-23d2b25963be None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:22.521485 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635cb1b-448d-4ce6-a8e2-23d2b25963be None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:22.522003 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635cb1b-448d-4ce6-a8e2-23d2b25963be None admin] PATH_INFO: `/v3/projects/0dc9df6fada54f72a35899fa21f03b20` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:22.523439 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635cb1b-448d-4ce6-a8e2-23d2b25963be None admin] RBAC: Authorizing `identity:get_project(project_id=0dc9df6fada54f72a35899fa21f03b20)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:22.525099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635cb1b-448d-4ce6-a8e2-23d2b25963be None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:22.527339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 201/434] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:03:22 2022] GET /identity/v3/projects/0dc9df6fada54f72a35899fa21f03b20 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:22.534399 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635cb1b-448d-4ce6-a8e2-23d2b25963be None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:22.571215 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:22.601314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0164cbc-f838-4b68-a876-f7a7d6d383b5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:22.642560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635cb1b-448d-4ce6-a8e2-23d2b25963be None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:22.644873 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d2658-3431-44b5-a4c0-87c2c76ae48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:22.647349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d2658-3431-44b5-a4c0-87c2c76ae48e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:22.647349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d2658-3431-44b5-a4c0-87c2c76ae48e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:22.647349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d2658-3431-44b5-a4c0-87c2c76ae48e None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:22.648496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d2658-3431-44b5-a4c0-87c2c76ae48e None admin] RBAC: Authorizing `identity:get_project(project_id=e7099ea37d874a7aa73b15453887dffc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:22.649965 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d2658-3431-44b5-a4c0-87c2c76ae48e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:22.652006 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 234/435] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:03:22 2022] GET /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:22.657297 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d2658-3431-44b5-a4c0-87c2c76ae48e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:22.671171 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:22.697951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a851b5-f458-4571-9e38-479a30dd01c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:22.699034 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a851b5-f458-4571-9e38-479a30dd01c6 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:22.699417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a851b5-f458-4571-9e38-479a30dd01c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:22.699822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a851b5-f458-4571-9e38-479a30dd01c6 None admin] PATH_INFO: `/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:22.701419 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a851b5-f458-4571-9e38-479a30dd01c6 None admin] RBAC: Authorizing `identity:get_group(group_id=04f5c67e865c4a30a1bd97093e65cbcd)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:22.702921 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a851b5-f458-4571-9e38-479a30dd01c6 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:22.704983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 202/436] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Jun 27 21:03:22 2022] GET /identity/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:22.713465 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a851b5-f458-4571-9e38-479a30dd01c6 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:22.795465 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d2658-3431-44b5-a4c0-87c2c76ae48e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:22.836638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb38839-93f7-494e-aa6d-c5c89109f3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:22.837513 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb38839-93f7-494e-aa6d-c5c89109f3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:22.837795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb38839-93f7-494e-aa6d-c5c89109f3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:22.838068 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb38839-93f7-494e-aa6d-c5c89109f3d0 None admin] PATH_INFO: `/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:22.839509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb38839-93f7-494e-aa6d-c5c89109f3d0 None admin] RBAC: Authorizing `identity:get_group(group_id=04f5c67e865c4a30a1bd97093e65cbcd)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:22.839644 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a851b5-f458-4571-9e38-479a30dd01c6 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:22.841030 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb38839-93f7-494e-aa6d-c5c89109f3d0 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:22.842729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:22.844577 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 235/437] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Jun 27 21:03:22 2022] GET /identity/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:22.845873 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc81d1d-0fa8-40f3-953d-bd78bfae1e44 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:22.846160 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc81d1d-0fa8-40f3-953d-bd78bfae1e44 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:22.846435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc81d1d-0fa8-40f3-953d-bd78bfae1e44 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:22.847696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 236/438] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:22.852237 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc81d1d-0fa8-40f3-953d-bd78bfae1e44 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:22.889664 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30122757-9f6a-46af-9896-6ebd542f3c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:22.890472 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30122757-9f6a-46af-9896-6ebd542f3c4e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:22.890760 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30122757-9f6a-46af-9896-6ebd542f3c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:22.891077 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30122757-9f6a-46af-9896-6ebd542f3c4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:22.892632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30122757-9f6a-46af-9896-6ebd542f3c4e None admin] RBAC: Adding query filter params (group.id=04f5c67e865c4a30a1bd97093e65cbcd, role.id=b3efb2b2e43c4af8b1cf36a37f1e05f9, scope.project.id=0dc9df6fada54f72a35899fa21f03b20) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:22.892632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30122757-9f6a-46af-9896-6ebd542f3c4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:22.893742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30122757-9f6a-46af-9896-6ebd542f3c4e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:22.900212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 203/439] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Jun 27 21:03:22 2022] GET /identity/v3/role_assignments?group.id=04f5c67e865c4a30a1bd97093e65cbcd&scope.project.id=0dc9df6fada54f72a35899fa21f03b20&role.id=b3efb2b2e43c4af8b1cf36a37f1e05f9 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:22.908592 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5855ae-216a-4705-956c-9e668ad102e6 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:22.908592 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5855ae-216a-4705-956c-9e668ad102e6 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:22.908592 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5855ae-216a-4705-956c-9e668ad102e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:22.921846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6b5855ae-216a-4705-956c-9e668ad102e6 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:22.923075 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6b5855ae-216a-4705-956c-9e668ad102e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:22.965388 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:23.004263 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc81d1d-0fa8-40f3-953d-bd78bfae1e44 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:23.004351 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 204/440] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:23.028608 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5855ae-216a-4705-956c-9e668ad102e6 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:23.039691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a9d66-8433-46ef-ac9c-8f86fbe2a926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:23.040764 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a9d66-8433-46ef-ac9c-8f86fbe2a926 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:23.041073 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a9d66-8433-46ef-ac9c-8f86fbe2a926 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:23.041353 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a9d66-8433-46ef-ac9c-8f86fbe2a926 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:23.042537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a9d66-8433-46ef-ac9c-8f86fbe2a926 None admin] RBAC: Adding query filter params (group.id=04f5c67e865c4a30a1bd97093e65cbcd, role.id=b1bcb7445ee54636ae7b29a2485cc201, scope.project.id=e7099ea37d874a7aa73b15453887dffc) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:23.042856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a9d66-8433-46ef-ac9c-8f86fbe2a926 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:23.044703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a9d66-8433-46ef-ac9c-8f86fbe2a926 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:23.053698 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 237/441] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Jun 27 21:03:22 2022] GET /identity/v3/role_assignments?group.id=04f5c67e865c4a30a1bd97093e65cbcd&scope.project.id=e7099ea37d874a7aa73b15453887dffc&role.id=b1bcb7445ee54636ae7b29a2485cc201 => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:23.148307 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:23.155216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5855ae-216a-4705-956c-9e668ad102e6 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:23.196258 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9bdeb-b8df-4fba-a0c7-9a06384d8ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:23.197108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9bdeb-b8df-4fba-a0c7-9a06384d8ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:23.197382 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9bdeb-b8df-4fba-a0c7-9a06384d8ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:23.197663 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9bdeb-b8df-4fba-a0c7-9a06384d8ba1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:23.210222 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d9bdeb-b8df-4fba-a0c7-9a06384d8ba1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:23.218867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-09d9bdeb-b8df-4fba-a0c7-9a06384d8ba1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:03:23.218867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:23.218867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:23.218867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:23.218867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:23.218867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:23.218867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:23.218867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:23.218867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:23.218867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:23.218867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:23.218867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:23.218867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:23.218867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:23.218867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:23.218867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:23.219774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:23.219774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:23.219774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:23.219774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:23.219774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:23.219774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:23.219774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:23.219774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:23.219774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:23.219774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:23.219774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:23.219774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:23.219774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:23.219774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:23.219774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:23.219774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:23.220558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:23.220558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:23.220558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:23.220558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:23.220558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:23.220558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:23.220558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:23.220558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:23.220558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:23.220558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:23.220558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:03:23.220558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:23.220558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 205/442] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:03:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:23.226710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a9d66-8433-46ef-ac9c-8f86fbe2a926 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:23.336304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a9d66-8433-46ef-ac9c-8f86fbe2a926 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:23.341492 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfee55-845c-4f18-9e5e-dbcefbdd131a None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:23.341776 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfee55-845c-4f18-9e5e-dbcefbdd131a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:23.342053 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfee55-845c-4f18-9e5e-dbcefbdd131a None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:23.343427 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 206/443] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:23.350426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a671842-8a69-4156-9f8d-e0889a838b6c None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:23.350710 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a671842-8a69-4156-9f8d-e0889a838b6c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:23.351829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a671842-8a69-4156-9f8d-e0889a838b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:23.373643 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-3a671842-8a69-4156-9f8d-e0889a838b6c None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:23.374877 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3a671842-8a69-4156-9f8d-e0889a838b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:23.379566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71553897-9656-46ad-8514-3d0b39b2c2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:23.380278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71553897-9656-46ad-8514-3d0b39b2c2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:23.380562 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71553897-9656-46ad-8514-3d0b39b2c2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:23.380843 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71553897-9656-46ad-8514-3d0b39b2c2d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:23.381601 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71553897-9656-46ad-8514-3d0b39b2c2d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:23.381915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71553897-9656-46ad-8514-3d0b39b2c2d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:23.383780 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71553897-9656-46ad-8514-3d0b39b2c2d6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:23.394004 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 238/444] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:23.399423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71553897-9656-46ad-8514-3d0b39b2c2d6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:23.402493 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:23.453894 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 207/445] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:03:23.477151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a671842-8a69-4156-9f8d-e0889a838b6c None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:23.521573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71553897-9656-46ad-8514-3d0b39b2c2d6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:23.554242 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770a669-601e-40d0-93ce-8f7dd0a811cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:23.554893 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770a669-601e-40d0-93ce-8f7dd0a811cb None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:23.555174 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770a669-601e-40d0-93ce-8f7dd0a811cb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:23.555445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770a669-601e-40d0-93ce-8f7dd0a811cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:23.564830 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:23.579697 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a671842-8a69-4156-9f8d-e0889a838b6c None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:23.615240 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0c0d2-707b-4d0b-8e44-9b3407591cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:23.615998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0c0d2-707b-4d0b-8e44-9b3407591cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:23.616278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0c0d2-707b-4d0b-8e44-9b3407591cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:23.616550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0c0d2-707b-4d0b-8e44-9b3407591cb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:23.621573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770a669-601e-40d0-93ce-8f7dd0a811cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:23.623231 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770a669-601e-40d0-93ce-8f7dd0a811cb None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:23.625140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:23.630286 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d0c0d2-707b-4d0b-8e44-9b3407591cb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:23.638304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-b7d0c0d2-707b-4d0b-8e44-9b3407591cb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:23.638304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:23.638304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:23.638304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:23.638304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:23.638304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:23.638304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:23.638304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:23.638304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:23.638304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:23.638304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:23.638304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:23.638304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:23.638304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:23.638304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:23.638304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:23.639531 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:23.639531 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:23.639531 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:23.639531 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:23.639531 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:23.639531 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:23.639531 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:23.639531 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:23.639531 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:23.639531 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:23.639531 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:23.639531 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:23.639531 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:23.639531 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:23.639531 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:23.639531 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:23.640438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:23.640438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:23.640438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:23.640438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:23.640438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:23.640438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:23.640438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:23.640438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:23.640438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:23.640438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:23.640438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:23.640438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:23.640438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 208/446] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:23.645379 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0c0d2-707b-4d0b-8e44-9b3407591cb4 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:23.727738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 239/447] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 21:03:23.733185 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770a669-601e-40d0-93ce-8f7dd0a811cb None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:23.750214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0c0d2-707b-4d0b-8e44-9b3407591cb4 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:23.783781 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e152c0-99dd-4a79-ad98-58c2c53bb54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:23.784448 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e152c0-99dd-4a79-ad98-58c2c53bb54a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:23.784734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e152c0-99dd-4a79-ad98-58c2c53bb54a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:23.785013 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e152c0-99dd-4a79-ad98-58c2c53bb54a None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:23.785766 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e152c0-99dd-4a79-ad98-58c2c53bb54a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:23.786075 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e152c0-99dd-4a79-ad98-58c2c53bb54a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:23.787527 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e152c0-99dd-4a79-ad98-58c2c53bb54a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:23.795492 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 209/448] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:23.800887 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e152c0-99dd-4a79-ad98-58c2c53bb54a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:23.811348 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:23.832998 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770a669-601e-40d0-93ce-8f7dd0a811cb None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:23.834618 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:23.868300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1db97-0642-4930-9a46-d20398a023cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:23.869101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1db97-0642-4930-9a46-d20398a023cf None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:23.869376 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1db97-0642-4930-9a46-d20398a023cf None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:23.869648 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1db97-0642-4930-9a46-d20398a023cf None admin] PATH_INFO: `/v3/projects/e7099ea37d874a7aa73b15453887dffc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:23.870815 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1db97-0642-4930-9a46-d20398a023cf None admin] RBAC: Authorizing `identity:get_project(project_id=e7099ea37d874a7aa73b15453887dffc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:23.872291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1db97-0642-4930-9a46-d20398a023cf None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:23.873981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 240/449] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:03:23 2022] GET /identity/v3/projects/e7099ea37d874a7aa73b15453887dffc => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:23.879424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1db97-0642-4930-9a46-d20398a023cf None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:23.901093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e152c0-99dd-4a79-ad98-58c2c53bb54a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:23.934213 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eebf83-29d4-41ef-81cf-67fdedf43651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:23.934893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eebf83-29d4-41ef-81cf-67fdedf43651 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:23.935178 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eebf83-29d4-41ef-81cf-67fdedf43651 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:23.935457 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eebf83-29d4-41ef-81cf-67fdedf43651 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:23.980509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1db97-0642-4930-9a46-d20398a023cf None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:24.004520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eebf83-29d4-41ef-81cf-67fdedf43651 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:24.006083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eebf83-29d4-41ef-81cf-67fdedf43651 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:24.015598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b15d5-28fc-41ef-ac69-766facd5eaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:24.016509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b15d5-28fc-41ef-ac69-766facd5eaf2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:24.016651 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:24.016794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b15d5-28fc-41ef-ac69-766facd5eaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:24.017074 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b15d5-28fc-41ef-ac69-766facd5eaf2 None admin] PATH_INFO: `/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:24.018537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b15d5-28fc-41ef-ac69-766facd5eaf2 None admin] RBAC: Authorizing `identity:get_group(group_id=04f5c67e865c4a30a1bd97093e65cbcd)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:24.020107 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b15d5-28fc-41ef-ac69-766facd5eaf2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:24.021930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 241/450] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Jun 27 21:03:23 2022] GET /identity/v3/groups/04f5c67e865c4a30a1bd97093e65cbcd => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:24.027424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b15d5-28fc-41ef-ac69-766facd5eaf2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:24.066098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:24.108071 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 210/451] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 21:03:24.113661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eebf83-29d4-41ef-81cf-67fdedf43651 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:24.127213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b15d5-28fc-41ef-ac69-766facd5eaf2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:24.160800 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72850561-2671-4e2a-9463-9da1645d605d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:24.161461 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72850561-2671-4e2a-9463-9da1645d605d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:24.161744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72850561-2671-4e2a-9463-9da1645d605d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:24.162013 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72850561-2671-4e2a-9463-9da1645d605d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:24.162852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72850561-2671-4e2a-9463-9da1645d605d None admin] RBAC: Adding query filter params (group.id=04f5c67e865c4a30a1bd97093e65cbcd, role.id=b3efb2b2e43c4af8b1cf36a37f1e05f9, scope.project.id=e7099ea37d874a7aa73b15453887dffc) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:24.163135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72850561-2671-4e2a-9463-9da1645d605d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:24.165621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72850561-2671-4e2a-9463-9da1645d605d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:24.172462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 242/452] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Jun 27 21:03:24 2022] GET /identity/v3/role_assignments?group.id=04f5c67e865c4a30a1bd97093e65cbcd&scope.project.id=e7099ea37d874a7aa73b15453887dffc&role.id=b3efb2b2e43c4af8b1cf36a37f1e05f9 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:24.217023 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eebf83-29d4-41ef-81cf-67fdedf43651 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:24.228287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:24.256736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f7cf9-d751-4c9e-9f04-a5eef2c32eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:24.257622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f7cf9-d751-4c9e-9f04-a5eef2c32eeb None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:24.257899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f7cf9-d751-4c9e-9f04-a5eef2c32eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:24.258171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f7cf9-d751-4c9e-9f04-a5eef2c32eeb None admin] PATH_INFO: `/v3/projects/2213409bed61490ba55498fcd7bd10f8` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:24.259407 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f7cf9-d751-4c9e-9f04-a5eef2c32eeb None admin] RBAC: Authorizing `identity:get_project(project_id=2213409bed61490ba55498fcd7bd10f8)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:24.260938 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f7cf9-d751-4c9e-9f04-a5eef2c32eeb None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:24.262719 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 211/453] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:03:24 2022] GET /identity/v3/projects/2213409bed61490ba55498fcd7bd10f8 => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:24.268439 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72850561-2671-4e2a-9463-9da1645d605d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:24.377187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72850561-2671-4e2a-9463-9da1645d605d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:24.378777 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:24.413270 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6b8b7a-4673-4467-8bfe-866a9078748e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:24.414135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b8b7a-4673-4467-8bfe-866a9078748e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:24.414440 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b8b7a-4673-4467-8bfe-866a9078748e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:24.414700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b8b7a-4673-4467-8bfe-866a9078748e None admin] PATH_INFO: `/v3/groups/56c3d159c98640d180177ccc434665f5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:24.415862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6b8b7a-4673-4467-8bfe-866a9078748e None admin] RBAC: Authorizing `identity:get_group(group_id=56c3d159c98640d180177ccc434665f5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:24.417303 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6b8b7a-4673-4467-8bfe-866a9078748e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:24.418913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 243/454] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Jun 27 21:03:24 2022] GET /identity/v3/groups/56c3d159c98640d180177ccc434665f5 => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:24.425031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f7cf9-d751-4c9e-9f04-a5eef2c32eeb None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:24.528239 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f7cf9-d751-4c9e-9f04-a5eef2c32eeb None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:24.562928 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6d209-fb32-4b22-bdfe-63fbb4765554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:24.563580 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6d209-fb32-4b22-bdfe-63fbb4765554 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:24.563895 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6d209-fb32-4b22-bdfe-63fbb4765554 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:24.564178 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6d209-fb32-4b22-bdfe-63fbb4765554 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:24.565033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6d209-fb32-4b22-bdfe-63fbb4765554 None admin] RBAC: Adding query filter params (group.id=56c3d159c98640d180177ccc434665f5, role.id=5c3c095bb53943eca95c4e049cd8cbf5, scope.project.id=2213409bed61490ba55498fcd7bd10f8) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:24.565346 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6d209-fb32-4b22-bdfe-63fbb4765554 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:24.566787 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6d209-fb32-4b22-bdfe-63fbb4765554 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:24.572734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 212/455] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Jun 27 21:03:24 2022] GET /identity/v3/role_assignments?group.id=56c3d159c98640d180177ccc434665f5&scope.project.id=2213409bed61490ba55498fcd7bd10f8&role.id=5c3c095bb53943eca95c4e049cd8cbf5 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:26.893483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c8817-0059-42df-8ffc-44a3c6dc85ad None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:26.894042 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c8817-0059-42df-8ffc-44a3c6dc85ad None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:26.894042 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c8817-0059-42df-8ffc-44a3c6dc85ad None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:26.895658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 244/456] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:26.900855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286ccd0-762c-4489-b742-fb0cdb0102bf None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:26.901261 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286ccd0-762c-4489-b742-fb0cdb0102bf None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:26.901426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286ccd0-762c-4489-b742-fb0cdb0102bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:26.918739 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-2286ccd0-762c-4489-b742-fb0cdb0102bf None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:26.919939 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2286ccd0-762c-4489-b742-fb0cdb0102bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:26.961083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:27.005925 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 213/457] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:27.052015 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c8817-0059-42df-8ffc-44a3c6dc85ad None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:27.078472 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcb500-6461-45c7-9ffe-838775d8e4dd None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:27.078787 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcb500-6461-45c7-9ffe-838775d8e4dd None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:27.079050 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcb500-6461-45c7-9ffe-838775d8e4dd None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:27.080585 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 214/458] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:27.085891 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6df15f-0454-4c3a-ad1d-d82782bab0a0 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:27.086176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6df15f-0454-4c3a-ad1d-d82782bab0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:27.086454 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6df15f-0454-4c3a-ad1d-d82782bab0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:27.113930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-7b6df15f-0454-4c3a-ad1d-d82782bab0a0 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:27.115143 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7b6df15f-0454-4c3a-ad1d-d82782bab0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:27.228298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c8817-0059-42df-8ffc-44a3c6dc85ad None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:27.230048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 215/459] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:03:27.231670 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99b98e-f864-4f89-afdc-45a4f84e210e None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:27.232095 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99b98e-f864-4f89-afdc-45a4f84e210e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:27.232501 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99b98e-f864-4f89-afdc-45a4f84e210e None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:27.234131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 216/460] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:27.245009 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a519a-1bb5-471c-bb8f-d7b38e09dac0 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:27.245471 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a519a-1bb5-471c-bb8f-d7b38e09dac0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:27.245891 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a519a-1bb5-471c-bb8f-d7b38e09dac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:27.263496 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-657a519a-1bb5-471c-bb8f-d7b38e09dac0 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:27.264967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-657a519a-1bb5-471c-bb8f-d7b38e09dac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:27.276449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7df53-dd97-4492-af38-d3b423285dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:27.277478 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7df53-dd97-4492-af38-d3b423285dac None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:27.277924 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7df53-dd97-4492-af38-d3b423285dac None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:27.278359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7df53-dd97-4492-af38-d3b423285dac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:27.291626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7df53-dd97-4492-af38-d3b423285dac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:27.293262 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7df53-dd97-4492-af38-d3b423285dac None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:27.312741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-62a7df53-dd97-4492-af38-d3b423285dac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:27.312741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:27.312741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:27.312741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:27.312741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:27.312741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:27.312741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:27.312741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:27.312741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:27.312741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:27.312741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:27.312741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:27.312741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:27.312741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:27.312741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:27.313590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:27.313590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:27.313590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:27.313590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:27.313590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:27.313590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:27.313590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:27.313590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:27.313590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:27.313590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:27.313590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:27.313590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:27.313590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:27.313590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:27.313590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:27.313590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:27.313590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:27.314683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:27.314683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:27.314683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:27.314683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:27.314683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 245/461] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:27.315202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7df53-dd97-4492-af38-d3b423285dac None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:27.342793 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:27.343709 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 217/462] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:03:27.344728 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a519a-1bb5-471c-bb8f-d7b38e09dac0 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:27.382226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:27.457373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7df53-dd97-4492-af38-d3b423285dac None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:27.472873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a519a-1bb5-471c-bb8f-d7b38e09dac0 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:27.496369 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d659fd-b945-4862-9ab8-d0b0808d8c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:27.497109 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d659fd-b945-4862-9ab8-d0b0808d8c75 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:27.497396 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d659fd-b945-4862-9ab8-d0b0808d8c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:27.497674 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d659fd-b945-4862-9ab8-d0b0808d8c75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:27.509247 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d659fd-b945-4862-9ab8-d0b0808d8c75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:27.509914 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271611c-1727-4861-beb9-8d6e99100380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:27.510738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d659fd-b945-4862-9ab8-d0b0808d8c75 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:27.511002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271611c-1727-4861-beb9-8d6e99100380 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:27.511644 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271611c-1727-4861-beb9-8d6e99100380 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:27.512083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271611c-1727-4861-beb9-8d6e99100380 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:27.512985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271611c-1727-4861-beb9-8d6e99100380 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:27.513457 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271611c-1727-4861-beb9-8d6e99100380 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:27.515006 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271611c-1727-4861-beb9-8d6e99100380 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:27.531809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-90d659fd-b945-4862-9ab8-d0b0808d8c75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:27.531809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:27.531809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:27.531809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:27.531809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:27.531809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:27.531809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:27.531809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:27.531809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:27.531809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:27.531809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:27.531809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:27.531809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:27.531809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:27.531809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:27.532681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:27.532681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:27.532681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:27.532681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:27.532681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:27.532681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:27.532681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:27.532681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:27.532681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:27.532681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:27.532681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:27.532681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:27.532681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:27.532681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:27.532681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:27.532681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:27.532681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:27.533931 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:27.533931 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:27.533931 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:27.533931 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:27.533931 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 246/464] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:27.533931 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d659fd-b945-4862-9ab8-d0b0808d8c75 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:27.534471 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 218/464] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:27.535490 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271611c-1727-4861-beb9-8d6e99100380 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:27.618555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:27.629988 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:27.646425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271611c-1727-4861-beb9-8d6e99100380 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:27.648695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d659fd-b945-4862-9ab8-d0b0808d8c75 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:27.684884 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2445c3-41a5-4cd7-b699-7c9199cb0eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:27.686051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2445c3-41a5-4cd7-b699-7c9199cb0eca None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:27.686507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2445c3-41a5-4cd7-b699-7c9199cb0eca None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:27.686990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2445c3-41a5-4cd7-b699-7c9199cb0eca None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:27.688056 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2445c3-41a5-4cd7-b699-7c9199cb0eca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:27.690770 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2445c3-41a5-4cd7-b699-7c9199cb0eca None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:27.696861 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5155b-8ceb-41da-bf07-1881a5429c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:27.698047 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5155b-8ceb-41da-bf07-1881a5429c8a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:27.698332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5155b-8ceb-41da-bf07-1881a5429c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:27.698612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5155b-8ceb-41da-bf07-1881a5429c8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:27.714573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5155b-8ceb-41da-bf07-1881a5429c8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:27.716674 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5155b-8ceb-41da-bf07-1881a5429c8a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:27.717951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 219/465] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Jun 27 21:03:27 2022] POST /identity/v3/users => generated 246 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:27.718968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2445c3-41a5-4cd7-b699-7c9199cb0eca None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:27.763165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-a8e5155b-8ceb-41da-bf07-1881a5429c8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:27.763165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:27.763165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:27.763165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:27.763165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:27.763165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:27.763165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:27.763165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:27.763165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:27.763165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:27.763165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:27.763165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:27.763165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:27.763165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:27.763165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:27.764180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:27.764180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:27.764180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:27.764180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:27.764180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:27.764180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:27.764180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:27.764180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:27.764180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:27.764180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:27.764180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:27.764180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:27.764180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:27.764180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:27.764180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:27.764180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:27.764180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:27.765227 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:27.765227 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:27.765227 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:27.765227 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:27.766965 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 247/466] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:27.772885 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5155b-8ceb-41da-bf07-1881a5429c8a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:27.836586 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2445c3-41a5-4cd7-b699-7c9199cb0eca None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:27.875096 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06d011-ee0d-4d73-84ef-7759c9f703fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:27.876087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06d011-ee0d-4d73-84ef-7759c9f703fb None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:27.876504 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06d011-ee0d-4d73-84ef-7759c9f703fb None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:27.877038 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06d011-ee0d-4d73-84ef-7759c9f703fb None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:27.878011 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea06d011-ee0d-4d73-84ef-7759c9f703fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:27.878465 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea06d011-ee0d-4d73-84ef-7759c9f703fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:27.880133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea06d011-ee0d-4d73-84ef-7759c9f703fb None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:27.894346 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 220/467] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:27.902620 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5155b-8ceb-41da-bf07-1881a5429c8a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:27.904386 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06d011-ee0d-4d73-84ef-7759c9f703fb None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:27.914931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:27.939298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8174851-a681-4878-83e7-dadda1a7b63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:27.940477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8174851-a681-4878-83e7-dadda1a7b63a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:27.940961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8174851-a681-4878-83e7-dadda1a7b63a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:27.941452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8174851-a681-4878-83e7-dadda1a7b63a None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:27.942301 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8174851-a681-4878-83e7-dadda1a7b63a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:27.942686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8174851-a681-4878-83e7-dadda1a7b63a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:27.944250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8174851-a681-4878-83e7-dadda1a7b63a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:27.946639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:27.977970 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 248/468] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:27.977970 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de78a5-6d68-4c41-8502-eead3bd9928a None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:27.977970 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de78a5-6d68-4c41-8502-eead3bd9928a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:27.977970 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de78a5-6d68-4c41-8502-eead3bd9928a None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:27.977970 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 249/469] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:27.984157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698c7d4-8435-48f2-97a0-4de36d3458ee None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:27.984564 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698c7d4-8435-48f2-97a0-4de36d3458ee None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:27.985044 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698c7d4-8435-48f2-97a0-4de36d3458ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:28.002564 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-1698c7d4-8435-48f2-97a0-4de36d3458ee None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:28.003839 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1698c7d4-8435-48f2-97a0-4de36d3458ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:28.044532 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06d011-ee0d-4d73-84ef-7759c9f703fb None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:28.085416 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7f144-8f27-4e5b-a441-f92d86cabdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:28.086446 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 250/470] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:28.086812 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7f144-8f27-4e5b-a441-f92d86cabdd4 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:28.087096 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698c7d4-8435-48f2-97a0-4de36d3458ee None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:28.087383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7f144-8f27-4e5b-a441-f92d86cabdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:28.087819 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7f144-8f27-4e5b-a441-f92d86cabdd4 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:28.088741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc7f144-8f27-4e5b-a441-f92d86cabdd4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:28.090310 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc7f144-8f27-4e5b-a441-f92d86cabdd4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:28.127723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 221/471] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Jun 27 21:03:27 2022] POST /identity/v3/users => generated 248 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:28.128885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7f144-8f27-4e5b-a441-f92d86cabdd4 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:28.220628 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698c7d4-8435-48f2-97a0-4de36d3458ee None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:28.239498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:28.241244 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:28.260073 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31f6f5-fa3a-4d07-be77-a51e7698e1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:28.261069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31f6f5-fa3a-4d07-be77-a51e7698e1dd None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:28.261492 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31f6f5-fa3a-4d07-be77-a51e7698e1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:28.261913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31f6f5-fa3a-4d07-be77-a51e7698e1dd None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:28.262781 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31f6f5-fa3a-4d07-be77-a51e7698e1dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:28.264477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31f6f5-fa3a-4d07-be77-a51e7698e1dd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:28.291423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 251/472] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Jun 27 21:03:28 2022] POST /identity/v3/users => generated 248 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:28.341701 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7f144-8f27-4e5b-a441-f92d86cabdd4 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:28.412450 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59890836-84e4-4c73-88b0-767f4bc0b4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:28.413316 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59890836-84e4-4c73-88b0-767f4bc0b4ca None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:28.413606 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59890836-84e4-4c73-88b0-767f4bc0b4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:28.413886 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59890836-84e4-4c73-88b0-767f4bc0b4ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:28.426650 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59890836-84e4-4c73-88b0-767f4bc0b4ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:28.428195 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59890836-84e4-4c73-88b0-767f4bc0b4ca None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:28.448902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-59890836-84e4-4c73-88b0-767f4bc0b4ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:28.448902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:28.448902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:28.448902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:28.448902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:28.448902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:28.448902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:28.448902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:28.448902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:28.448902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:28.448902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:28.448902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:28.448902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:28.448902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:28.448902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:28.449798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:28.449798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:28.449798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:28.449798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:28.449798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:28.449798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:28.449798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:28.449798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:28.449798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:28.449798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:28.449798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:28.449798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:28.449798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:28.449798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:28.449798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:28.449798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:28.449798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:28.450607 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:28.450607 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:28.450607 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:28.450607 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:28.450607 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 222/473] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:28.457333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31f6f5-fa3a-4d07-be77-a51e7698e1dd None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:28.579156 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31f6f5-fa3a-4d07-be77-a51e7698e1dd None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:28.595674 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f5214-6865-407f-b34c-284ffa41c5e7 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:28.595973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f5214-6865-407f-b34c-284ffa41c5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:28.596169 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f5214-6865-407f-b34c-284ffa41c5e7 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:28.597571 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 223/474] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:28.602753 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70f18a-d2ff-4053-a34c-24322bbfbe4a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:28.603982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70f18a-d2ff-4053-a34c-24322bbfbe4a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:28.604487 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70f18a-d2ff-4053-a34c-24322bbfbe4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:28.620404 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec8667-436d-4f25-af53-9c8156cc5d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:28.621700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec8667-436d-4f25-af53-9c8156cc5d5b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:28.622183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec8667-436d-4f25-af53-9c8156cc5d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:28.622653 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec8667-436d-4f25-af53-9c8156cc5d5b None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:28.623627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec8667-436d-4f25-af53-9c8156cc5d5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:28.624134 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec8667-436d-4f25-af53-9c8156cc5d5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:28.625056 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ca70f18a-d2ff-4053-a34c-24322bbfbe4a None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:28.625986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec8667-436d-4f25-af53-9c8156cc5d5b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:28.626312 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ca70f18a-d2ff-4053-a34c-24322bbfbe4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:28.641396 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 252/475] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:28.647831 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec8667-436d-4f25-af53-9c8156cc5d5b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:28.687914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:28.716286 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 224/476] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:03:28.739394 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70f18a-d2ff-4053-a34c-24322bbfbe4a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:28.765400 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec8667-436d-4f25-af53-9c8156cc5d5b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:28.803989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:28.804771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4df2b-da5d-468f-a87a-291871eb111c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:28.805706 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4df2b-da5d-468f-a87a-291871eb111c None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:28.806170 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4df2b-da5d-468f-a87a-291871eb111c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:28.806645 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4df2b-da5d-468f-a87a-291871eb111c None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:28.807558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4df2b-da5d-468f-a87a-291871eb111c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:28.809309 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4df2b-da5d-468f-a87a-291871eb111c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:28.835766 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 253/477] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Jun 27 21:03:28 2022] POST /identity/v3/users => generated 249 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:03:28.927394 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70f18a-d2ff-4053-a34c-24322bbfbe4a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:28.985570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e67e4-7e10-4c72-82d4-11b02f1354ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:28.986345 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e67e4-7e10-4c72-82d4-11b02f1354ae None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:28.986630 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e67e4-7e10-4c72-82d4-11b02f1354ae None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:28.986906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e67e4-7e10-4c72-82d4-11b02f1354ae None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:28.987694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e67e4-7e10-4c72-82d4-11b02f1354ae None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:28.989416 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e67e4-7e10-4c72-82d4-11b02f1354ae None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:29.003523 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.sql.core [None req-094e67e4-7e10-4c72-82d4-11b02f1354ae None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 27 21:03:29.003523 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 27 21:03:29.003523 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [parameters: {'id': '4f6d2846ff0043b489557b836936b1cc', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 27 21:03:29.003523 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80401) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 27 21:03:29.021000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-094e67e4-7e10-4c72-82d4-11b02f1354ae None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 27 21:03:29.021000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:29.021000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 21:03:29.021000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 21:03:29.021000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 21:03:29.021000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 21:03:29.021000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 21:03:29.021000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 21:03:29.021000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 21:03:29.021000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application conn.query(q) Jun 27 21:03:29.021000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 21:03:29.021000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 21:03:29.021000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 21:03:29.021000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application result.read() Jun 27 21:03:29.021000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 21:03:29.021960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 21:03:29.021960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 21:03:29.021960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 21:03:29.021960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 21:03:29.021960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 21:03:29.021960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 21:03:29.021960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 21:03:29.021960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 27 21:03:29.021960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:29.021960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 27 21:03:29.021960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:29.021960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:29.021960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 27 21:03:29.021960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 27 21:03:29.021960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 27 21:03:29.021960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return ref.to_dict() Jun 27 21:03:29.021960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 21:03:29.021960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application next(self.gen) Jun 27 21:03:29.023358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 27 21:03:29.023358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application yield resource Jun 27 21:03:29.023358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 21:03:29.023358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application next(self.gen) Jun 27 21:03:29.023358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 27 21:03:29.023358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self.session.rollback() Jun 27 21:03:29.023358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 27 21:03:29.023358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self.force_reraise() Jun 27 21:03:29.023358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 27 21:03:29.023358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise self.value Jun 27 21:03:29.023358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 27 21:03:29.023358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 27 21:03:29.023358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 27 21:03:29.023358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application session.commit() Jun 27 21:03:29.023358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 27 21:03:29.023358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 27 21:03:29.023358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 27 21:03:29.024467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._prepare_impl() Jun 27 21:03:29.024467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 27 21:03:29.024467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self.session.flush() Jun 27 21:03:29.024467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 27 21:03:29.024467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._flush(objects) Jun 27 21:03:29.024467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 27 21:03:29.024467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 27 21:03:29.024467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 27 21:03:29.024467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application compat.raise_( Jun 27 21:03:29.024467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 21:03:29.024467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception Jun 27 21:03:29.024467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 27 21:03:29.024467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application flush_context.execute() Jun 27 21:03:29.024467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 27 21:03:29.024467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rec.execute(self) Jun 27 21:03:29.024467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 27 21:03:29.024467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 27 21:03:29.025319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 27 21:03:29.025319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 27 21:03:29.025319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 27 21:03:29.025319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 27 21:03:29.025319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 27 21:03:29.025319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 27 21:03:29.025319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 27 21:03:29.025319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 27 21:03:29.025319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 27 21:03:29.025319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 27 21:03:29.025319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 27 21:03:29.025319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 27 21:03:29.025319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 27 21:03:29.025319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 27 21:03:29.025319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 21:03:29.025319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception Jun 27 21:03:29.026101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 21:03:29.026101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 21:03:29.026101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 21:03:29.026101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 21:03:29.026101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 21:03:29.026101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 21:03:29.026101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 21:03:29.026101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application conn.query(q) Jun 27 21:03:29.026101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 21:03:29.026101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 21:03:29.026101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 21:03:29.026101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application result.read() Jun 27 21:03:29.026101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 21:03:29.026101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 21:03:29.026101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 21:03:29.026101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 21:03:29.026867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 21:03:29.026867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 21:03:29.026867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 21:03:29.026867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 21:03:29.026867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 27 21:03:29.026867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 27 21:03:29.026867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [parameters: {'id': '4f6d2846ff0043b489557b836936b1cc', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 27 21:03:29.026867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 27 21:03:29.026867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:29.026867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:29.026867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:29.026867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:29.026867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:29.026867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:29.026867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:29.026867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:29.029544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:29.029544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:29.029544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:29.029544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:29.029544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:29.029544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:29.029544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 27 21:03:29.029544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 27 21:03:29.029544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:29.029544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:29.029544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 27 21:03:29.029544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 27 21:03:29.029544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 27 21:03:29.029544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 27 21:03:29.029544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 27 21:03:29.029544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:29.031464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 225/478] 10.222.0.27 () {60 vars in 1131 bytes} [Mon Jun 27 21:03:28 2022] POST /identity/v3/roles => generated 144 bytes in 284 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:29.031596 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4df2b-da5d-468f-a87a-291871eb111c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:29.135385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:29.152979 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4df2b-da5d-468f-a87a-291871eb111c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:29.190789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d879baf-e79b-4d9f-aaec-290ba55f0596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:29.191645 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d879baf-e79b-4d9f-aaec-290ba55f0596 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:29.191981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d879baf-e79b-4d9f-aaec-290ba55f0596 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:29.192260 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d879baf-e79b-4d9f-aaec-290ba55f0596 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:29.202734 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d879baf-e79b-4d9f-aaec-290ba55f0596 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:29.211196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-0d879baf-e79b-4d9f-aaec-290ba55f0596 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:03:29.211196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:29.211196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:29.211196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:29.211196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:29.211196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:29.211196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:29.211196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:29.211196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:29.211196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:29.211196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:29.211196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:29.211196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:29.211196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:29.211196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:29.211196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:29.212058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:29.212058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:29.212058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:29.212058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:29.212058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:29.212058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:29.212058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:29.212058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:29.212058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:29.212058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:29.212058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:29.212058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:29.212058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:29.212058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:29.212058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:29.212058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:29.213069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:29.213069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:29.213069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:29.213069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:29.213069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:29.213069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:29.213069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:29.213069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:29.213069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:29.213069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:29.213069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:03:29.213069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:29.213069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 254/479] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:03:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:29.218028 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e67e4-7e10-4c72-82d4-11b02f1354ae None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:29.326966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e67e4-7e10-4c72-82d4-11b02f1354ae None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:29.336646 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:29.362515 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbb7ad-ec47-4eb5-af13-35685e7b4e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:29.363193 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbb7ad-ec47-4eb5-af13-35685e7b4e79 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:29.363471 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbb7ad-ec47-4eb5-af13-35685e7b4e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:29.363775 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbb7ad-ec47-4eb5-af13-35685e7b4e79 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:29.364577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbb7ad-ec47-4eb5-af13-35685e7b4e79 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:29.364885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbb7ad-ec47-4eb5-af13-35685e7b4e79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:29.366262 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbb7ad-ec47-4eb5-af13-35685e7b4e79 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:29.375618 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 226/480] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:30.063828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b49c6c-8fb6-4c2d-aa60-e223837895b4 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:30.064416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b49c6c-8fb6-4c2d-aa60-e223837895b4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:30.064416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b49c6c-8fb6-4c2d-aa60-e223837895b4 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:30.066148 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 255/481] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:30.071558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce3bd9-4c56-40a4-93a2-04c09fe6ff23 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:30.072077 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce3bd9-4c56-40a4-93a2-04c09fe6ff23 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:30.072194 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce3bd9-4c56-40a4-93a2-04c09fe6ff23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:30.089784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-4fce3bd9-4c56-40a4-93a2-04c09fe6ff23 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:30.090968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4fce3bd9-4c56-40a4-93a2-04c09fe6ff23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:30.167982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 227/482] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:03:30.196947 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b49c6c-8fb6-4c2d-aa60-e223837895b4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:30.312566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b49c6c-8fb6-4c2d-aa60-e223837895b4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:30.347189 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e315c86-d549-4b97-b03c-d6324a7e48b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:30.348095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e315c86-d549-4b97-b03c-d6324a7e48b0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:30.348375 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e315c86-d549-4b97-b03c-d6324a7e48b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:30.348650 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e315c86-d549-4b97-b03c-d6324a7e48b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:30.359212 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e315c86-d549-4b97-b03c-d6324a7e48b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:30.367132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4e315c86-d549-4b97-b03c-d6324a7e48b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:30.367132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:30.367132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:30.367132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:30.367132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:30.367132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:30.367132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:30.367132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:30.367132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:30.367132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:30.367132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:30.367132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:30.367132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:30.367132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:30.367132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:30.367132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:30.368007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:30.368007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:30.368007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:30.368007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:30.368007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:30.368007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:30.368007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:30.368007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:30.368007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:30.368007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:30.368007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:30.368007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:30.368007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:30.368007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:30.368007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:30.368007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:30.368872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:30.368872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:30.368872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:30.368872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:30.368872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:30.368872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:30.368872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:30.368872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:30.368872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:30.368872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:30.368872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:30.368872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:30.368872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 256/483] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:30.376291 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce3bd9-4c56-40a4-93a2-04c09fe6ff23 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:30.510426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:30.513548 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce3bd9-4c56-40a4-93a2-04c09fe6ff23 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:30.561308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c5eec-8bbe-4361-a582-de175d53d954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:30.561992 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c5eec-8bbe-4361-a582-de175d53d954 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:30.562279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c5eec-8bbe-4361-a582-de175d53d954 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:30.562556 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c5eec-8bbe-4361-a582-de175d53d954 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:30.563362 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c5eec-8bbe-4361-a582-de175d53d954 None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:30.563631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c5eec-8bbe-4361-a582-de175d53d954 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:30.565063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c5eec-8bbe-4361-a582-de175d53d954 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:30.582451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 228/484] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:30.587918 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e315c86-d549-4b97-b03c-d6324a7e48b0 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:30.602842 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:30.705618 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e315c86-d549-4b97-b03c-d6324a7e48b0 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:30.744572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a476a58-99aa-4527-9fc4-5611eee9d9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:30.745381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a476a58-99aa-4527-9fc4-5611eee9d9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:30.745656 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a476a58-99aa-4527-9fc4-5611eee9d9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:30.745923 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a476a58-99aa-4527-9fc4-5611eee9d9a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:30.757016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a476a58-99aa-4527-9fc4-5611eee9d9a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:30.759162 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a476a58-99aa-4527-9fc4-5611eee9d9a9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:30.759646 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73e254-da55-429d-8a28-7687f1a1d827 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:30.759948 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73e254-da55-429d-8a28-7687f1a1d827 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:30.760353 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73e254-da55-429d-8a28-7687f1a1d827 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:30.762290 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 229/485] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:30.767242 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39228610-2dcc-4dff-8459-d864f83313fa None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:30.767651 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39228610-2dcc-4dff-8459-d864f83313fa None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:30.767941 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39228610-2dcc-4dff-8459-d864f83313fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:30.784294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-6a476a58-99aa-4527-9fc4-5611eee9d9a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:30.784294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:30.784294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:30.784294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:30.784294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:30.784294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:30.784294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:30.784294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:30.784294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:30.784294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:30.784294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:30.784294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:30.784294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:30.784294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:30.784294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:30.785140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:30.785140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:30.785140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:30.785140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:30.785140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:30.785140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:30.785140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:30.785140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:30.785140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:30.785140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:30.785140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:30.785140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:30.785140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:30.785140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:30.785140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:30.785140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:30.785140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:30.785965 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:30.785965 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:30.785965 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:30.785965 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:30.795426 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 257/486] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:30.803490 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-39228610-2dcc-4dff-8459-d864f83313fa None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:30.804943 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-39228610-2dcc-4dff-8459-d864f83313fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:30.808409 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a476a58-99aa-4527-9fc4-5611eee9d9a9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:30.910698 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:30.937397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 230/487] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:03:30.961609 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39228610-2dcc-4dff-8459-d864f83313fa None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:31.000601 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a476a58-99aa-4527-9fc4-5611eee9d9a9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:31.065332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b384d8-5a1b-4d33-822f-b8299090283f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:31.066842 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b384d8-5a1b-4d33-822f-b8299090283f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:31.068417 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b384d8-5a1b-4d33-822f-b8299090283f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:31.068891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b384d8-5a1b-4d33-822f-b8299090283f None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.069862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b384d8-5a1b-4d33-822f-b8299090283f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:31.070535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b384d8-5a1b-4d33-822f-b8299090283f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:31.073534 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b384d8-5a1b-4d33-822f-b8299090283f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:31.095481 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 258/488] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:31.100301 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233f0f2-c878-4150-97c0-ce8859e4f67e None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:31.100301 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233f0f2-c878-4150-97c0-ce8859e4f67e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:31.100301 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233f0f2-c878-4150-97c0-ce8859e4f67e None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.100301 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 259/489] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:31.103715 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233f0f2-c878-4150-97c0-ce8859e4f67e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:31.130041 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:31.143790 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39228610-2dcc-4dff-8459-d864f83313fa None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:31.186569 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c14c9b4-d6d1-40b7-b89a-e4660ff1e84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:31.187500 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c14c9b4-d6d1-40b7-b89a-e4660ff1e84e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:31.187936 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c14c9b4-d6d1-40b7-b89a-e4660ff1e84e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:31.188311 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c14c9b4-d6d1-40b7-b89a-e4660ff1e84e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.200242 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c14c9b4-d6d1-40b7-b89a-e4660ff1e84e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:31.209065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-2c14c9b4-d6d1-40b7-b89a-e4660ff1e84e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:31.209065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:31.209065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:31.209065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:31.209065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:31.209065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:31.209065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:31.209065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:31.209065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:31.209065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:31.209065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:31.209065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:31.209065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:31.209065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:31.209065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:31.209065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:31.210156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:31.210156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:31.210156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:31.210156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:31.210156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:31.210156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:31.210156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:31.210156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:31.210156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:31.210156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:31.210156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:31.210156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:31.210156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:31.210156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:31.210156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:31.210156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:31.210935 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:31.210935 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:31.210935 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:31.210935 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:31.210935 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:31.210935 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:31.210935 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:31.210935 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:31.210935 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:31.210935 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:31.210935 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:31.210935 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:31.211520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 231/490] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:31.213330 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd91ec7-f711-4910-b58f-ad2a37eb255a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:31.213716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd91ec7-f711-4910-b58f-ad2a37eb255a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:31.214027 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd91ec7-f711-4910-b58f-ad2a37eb255a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.232861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5fd91ec7-f711-4910-b58f-ad2a37eb255a None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:31.234158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5fd91ec7-f711-4910-b58f-ad2a37eb255a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:31.234812 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233f0f2-c878-4150-97c0-ce8859e4f67e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:31.262509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:31.280109 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff62b8-5cb1-44d4-a46d-5c6eb102532b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:31.281692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff62b8-5cb1-44d4-a46d-5c6eb102532b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:31.282174 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff62b8-5cb1-44d4-a46d-5c6eb102532b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:31.282616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff62b8-5cb1-44d4-a46d-5c6eb102532b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.293024 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff62b8-5cb1-44d4-a46d-5c6eb102532b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:31.295022 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff62b8-5cb1-44d4-a46d-5c6eb102532b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:31.314939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-cdff62b8-5cb1-44d4-a46d-5c6eb102532b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:31.314939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:31.314939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:31.314939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:31.314939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:31.314939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:31.314939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:31.314939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:31.314939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:31.314939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:31.314939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:31.314939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:31.314939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:31.314939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:31.314939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:31.314939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:31.315887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:31.315887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:31.315887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:31.315887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:31.315887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:31.315887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:31.315887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:31.315887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:31.315887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:31.315887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:31.315887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:31.315887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:31.315887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:31.315887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:31.315887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:31.315887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:31.315887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:31.315887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:31.316755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:31.316755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:31.316755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:31.316755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:31.316755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:31.316755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:31.316755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:31.316755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:31.316755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:31.316755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:31.317830 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 232/491] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:31.318815 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd91ec7-f711-4910-b58f-ad2a37eb255a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:31.323783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 260/492] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Jun 27 21:03:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:31.325957 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff62b8-5cb1-44d4-a46d-5c6eb102532b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:31.439450 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:31.448484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff62b8-5cb1-44d4-a46d-5c6eb102532b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:31.467863 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd91ec7-f711-4910-b58f-ad2a37eb255a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:31.486700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d1a55-73ce-43a8-85b3-2fbb98d2ec23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:31.487548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d1a55-73ce-43a8-85b3-2fbb98d2ec23 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:31.487956 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d1a55-73ce-43a8-85b3-2fbb98d2ec23 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:31.488402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d1a55-73ce-43a8-85b3-2fbb98d2ec23 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.489432 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d1a55-73ce-43a8-85b3-2fbb98d2ec23 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:31.489873 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d1a55-73ce-43a8-85b3-2fbb98d2ec23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:31.491637 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d1a55-73ce-43a8-85b3-2fbb98d2ec23 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:31.509458 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544efe5-1db9-4317-a860-218932cbceba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:31.510362 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544efe5-1db9-4317-a860-218932cbceba None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:31.510925 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544efe5-1db9-4317-a860-218932cbceba None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:31.511372 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544efe5-1db9-4317-a860-218932cbceba None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.512282 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544efe5-1db9-4317-a860-218932cbceba None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:31.512730 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544efe5-1db9-4317-a860-218932cbceba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:31.514265 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544efe5-1db9-4317-a860-218932cbceba None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:31.524131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 233/493] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:31.525164 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544efe5-1db9-4317-a860-218932cbceba None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:31.539975 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 261/494] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Jun 27 21:03:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:31.539975 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d1a55-73ce-43a8-85b3-2fbb98d2ec23 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:31.580833 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:31.636247 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544efe5-1db9-4317-a860-218932cbceba None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:31.643922 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d1a55-73ce-43a8-85b3-2fbb98d2ec23 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:31.673769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e98509-c1c3-491e-aa6f-9f2158ceef3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:31.674952 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e98509-c1c3-491e-aa6f-9f2158ceef3b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:31.675413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e98509-c1c3-491e-aa6f-9f2158ceef3b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:31.675986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e98509-c1c3-491e-aa6f-9f2158ceef3b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.678928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97aed1-8535-4b6a-a976-feb400a31f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:31.679769 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97aed1-8535-4b6a-a976-feb400a31f10 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:31.680055 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97aed1-8535-4b6a-a976-feb400a31f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:31.680336 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97aed1-8535-4b6a-a976-feb400a31f10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.689886 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e98509-c1c3-491e-aa6f-9f2158ceef3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:31.694656 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97aed1-8535-4b6a-a976-feb400a31f10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:31.696199 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97aed1-8535-4b6a-a976-feb400a31f10 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:31.699300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-d6e98509-c1c3-491e-aa6f-9f2158ceef3b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:31.699300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:31.699300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:31.699300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:31.699300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:31.699300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:31.699300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:31.699300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:31.699300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:31.699300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:31.699300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:31.699300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:31.699300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:31.699300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:31.699300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:31.699300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:31.700207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:31.700207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:31.700207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:31.700207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:31.700207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:31.700207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:31.700207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:31.700207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:31.700207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:31.700207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:31.700207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:31.700207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:31.700207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:31.700207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:31.700207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:31.700207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:31.701029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:31.701029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:31.701029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:31.701029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:31.701029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:31.701029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:31.701029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:31.701029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:31.701029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:31.701029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:31.701029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:31.701029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:31.701617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 234/495] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:31.702828 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be8c14-f36a-4e63-897e-26e72d81442f None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:31.703117 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be8c14-f36a-4e63-897e-26e72d81442f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:31.703400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be8c14-f36a-4e63-897e-26e72d81442f None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.707717 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 235/496] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:31.710104 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be8c14-f36a-4e63-897e-26e72d81442f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:31.714682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:31.721629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-1a97aed1-8535-4b6a-a976-feb400a31f10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:31.721629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:31.721629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:31.721629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:31.721629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:31.721629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:31.721629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:31.721629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:31.721629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:31.721629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:31.721629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:31.721629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:31.721629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:31.721629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:31.721629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:31.722484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:31.722484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:31.722484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:31.722484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:31.722484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:31.722484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:31.722484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:31.722484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:31.722484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:31.722484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:31.722484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:31.722484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:31.722484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:31.722484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:31.722484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:31.722484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:31.722484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:31.723315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:31.723315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:31.723315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:31.723315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:31.727704 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 262/497] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:31.728565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97aed1-8535-4b6a-a976-feb400a31f10 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:31.826162 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be8c14-f36a-4e63-897e-26e72d81442f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:31.826644 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:31.838141 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97aed1-8535-4b6a-a976-feb400a31f10 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:31.868037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cca89d-0464-447c-a5c6-91baa0169eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:31.868723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cca89d-0464-447c-a5c6-91baa0169eef None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:31.868996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cca89d-0464-447c-a5c6-91baa0169eef None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:31.869281 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cca89d-0464-447c-a5c6-91baa0169eef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.874757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ab92d-2995-46ed-a829-ebc394d86ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:31.875696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ab92d-2995-46ed-a829-ebc394d86ddc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:31.876173 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ab92d-2995-46ed-a829-ebc394d86ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:31.876659 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ab92d-2995-46ed-a829-ebc394d86ddc None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.877671 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ab92d-2995-46ed-a829-ebc394d86ddc None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:31.878174 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ab92d-2995-46ed-a829-ebc394d86ddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:31.879792 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ab92d-2995-46ed-a829-ebc394d86ddc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:31.888691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 263/498] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:31.890185 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410974d1-32e0-4af1-b235-8f32b86459df None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:31.890537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410974d1-32e0-4af1-b235-8f32b86459df None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:31.890897 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410974d1-32e0-4af1-b235-8f32b86459df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.908602 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-410974d1-32e0-4af1-b235-8f32b86459df None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:31.910533 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-410974d1-32e0-4af1-b235-8f32b86459df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:31.948731 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cca89d-0464-447c-a5c6-91baa0169eef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:31.950140 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cca89d-0464-447c-a5c6-91baa0169eef None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:31.953678 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:31.987656 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 264/499] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:31.987656 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410974d1-32e0-4af1-b235-8f32b86459df None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.073408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 236/500] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:32.074526 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cca89d-0464-447c-a5c6-91baa0169eef None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.090446 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:32.108633 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410974d1-32e0-4af1-b235-8f32b86459df None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:32.142504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04fb577-4c5b-4940-9531-4abcd032df8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:32.143168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fb577-4c5b-4940-9531-4abcd032df8a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.143448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fb577-4c5b-4940-9531-4abcd032df8a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.143717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fb577-4c5b-4940-9531-4abcd032df8a None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.144451 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fb577-4c5b-4940-9531-4abcd032df8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:32.144754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fb577-4c5b-4940-9531-4abcd032df8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:32.146135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fb577-4c5b-4940-9531-4abcd032df8a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:32.157035 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 265/501] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:32.157804 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04fb577-4c5b-4940-9531-4abcd032df8a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.174561 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cca89d-0464-447c-a5c6-91baa0169eef None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:32.177154 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:32.208776 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f41be3-ecce-437d-b3f4-c97507a634b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:32.209504 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f41be3-ecce-437d-b3f4-c97507a634b4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.209784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f41be3-ecce-437d-b3f4-c97507a634b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.210055 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f41be3-ecce-437d-b3f4-c97507a634b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.219853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f41be3-ecce-437d-b3f4-c97507a634b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:32.221352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f41be3-ecce-437d-b3f4-c97507a634b4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:32.238352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-f4f41be3-ecce-437d-b3f4-c97507a634b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:32.238352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:32.238352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:32.238352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:32.238352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:32.238352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:32.238352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:32.238352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:32.238352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:32.238352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:32.238352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:32.238352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:32.238352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:32.238352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:32.238352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:32.239171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:32.239171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:32.239171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:32.239171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:32.239171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:32.239171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:32.239171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:32.239171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:32.239171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:32.239171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:32.239171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:32.239171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:32.239171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:32.239171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:32.239171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:32.239171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:32.239171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:32.240039 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:32.240039 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:32.240039 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:32.240039 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:32.240039 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 237/502] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:32.240742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f41be3-ecce-437d-b3f4-c97507a634b4 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.262490 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04fb577-4c5b-4940-9531-4abcd032df8a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:32.296904 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d2a9d-4937-4542-ab74-a836e59e3855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:32.297701 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d2a9d-4937-4542-ab74-a836e59e3855 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.297976 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d2a9d-4937-4542-ab74-a836e59e3855 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.298247 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d2a9d-4937-4542-ab74-a836e59e3855 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.308658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d2a9d-4937-4542-ab74-a836e59e3855 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:32.316462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f65d2a9d-4937-4542-ab74-a836e59e3855 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:32.316462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:32.316462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:32.316462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:32.316462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:32.316462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:32.316462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:32.316462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:32.316462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:32.316462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:32.316462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:32.316462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:32.316462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:32.316462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:32.316462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:32.316462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:32.317392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:32.317392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:32.317392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:32.317392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:32.317392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:32.317392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:32.317392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:32.317392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:32.317392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:32.317392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:32.317392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:32.317392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:32.317392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:32.317392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:32.317392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:32.317392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:32.318275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:32.318275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:32.318275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:32.318275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:32.318275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:32.318275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:32.318275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:32.318275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:32.318275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:32.318275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:32.318275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:32.318275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:32.318275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 266/503] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:32.318940 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb1abc-b18a-4ae4-9f3e-3d17594f00fb None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.319156 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb1abc-b18a-4ae4-9f3e-3d17594f00fb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.319631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb1abc-b18a-4ae4-9f3e-3d17594f00fb None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.320675 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 267/504] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:32.321350 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb1abc-b18a-4ae4-9f3e-3d17594f00fb None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.324707 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:32.345986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f41be3-ecce-437d-b3f4-c97507a634b4 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:32.381174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4660ff3-1133-4cd6-83d3-eca6eab0ece9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:32.381978 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4660ff3-1133-4cd6-83d3-eca6eab0ece9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.382257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4660ff3-1133-4cd6-83d3-eca6eab0ece9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.382530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4660ff3-1133-4cd6-83d3-eca6eab0ece9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.392730 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4660ff3-1133-4cd6-83d3-eca6eab0ece9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:32.394185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4660ff3-1133-4cd6-83d3-eca6eab0ece9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:32.413300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-c4660ff3-1133-4cd6-83d3-eca6eab0ece9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:32.413300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:32.413300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:32.413300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:32.413300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:32.413300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:32.413300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:32.413300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:32.413300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:32.413300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:32.413300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:32.413300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:32.413300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:32.413300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:32.413300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:32.414133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:32.414133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:32.414133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:32.414133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:32.414133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:32.414133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:32.414133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:32.414133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:32.414133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:32.414133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:32.414133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:32.414133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:32.414133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:32.414133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:32.414133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:32.414133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:32.414133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:32.414964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:32.414964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:32.414964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:32.414964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:32.414964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 238/505] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:32.415678 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4660ff3-1133-4cd6-83d3-eca6eab0ece9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.425745 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:32.430881 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb1abc-b18a-4ae4-9f3e-3d17594f00fb None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:32.465144 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c02170-6dfa-48aa-a235-c39e60b18bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:32.465909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c02170-6dfa-48aa-a235-c39e60b18bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.466213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c02170-6dfa-48aa-a235-c39e60b18bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.466472 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c02170-6dfa-48aa-a235-c39e60b18bc3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.474063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c02170-6dfa-48aa-a235-c39e60b18bc3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:32.475530 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c02170-6dfa-48aa-a235-c39e60b18bc3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:32.489572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-41c02170-6dfa-48aa-a235-c39e60b18bc3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 27 21:03:32.489572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:32.489572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:32.489572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:32.489572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:32.489572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:32.489572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:32.489572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:32.489572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:32.489572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:32.489572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:32.489572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:32.489572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:32.489572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:32.489572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:32.489572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:32.490582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:32.490582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:32.490582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:32.490582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:32.490582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:32.490582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:32.490582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:32.490582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:32.490582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:32.490582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:32.490582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:32.490582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:32.490582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:32.490582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:32.490582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:32.490582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:32.490582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:32.490582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:32.491504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:32.491504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:32.491504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:32.491504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:32.491504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:32.491504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:32.491504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:32.491504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:32.491504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 27 21:03:32.491504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:32.491504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 268/506] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:32.492096 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c02170-6dfa-48aa-a235-c39e60b18bc3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.516032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4660ff3-1133-4cd6-83d3-eca6eab0ece9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:32.551282 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d80fba-3d08-4087-ae8b-3808989579e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:32.551978 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d80fba-3d08-4087-ae8b-3808989579e2 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.552260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d80fba-3d08-4087-ae8b-3808989579e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.552537 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d80fba-3d08-4087-ae8b-3808989579e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.553262 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d80fba-3d08-4087-ae8b-3808989579e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:32.553563 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d80fba-3d08-4087-ae8b-3808989579e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:32.553769 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:32.554963 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d80fba-3d08-4087-ae8b-3808989579e2 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:32.566541 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 239/507] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:32.567766 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171021b-1fd4-400f-b059-676d282b7b87 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.568056 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171021b-1fd4-400f-b059-676d282b7b87 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.568324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171021b-1fd4-400f-b059-676d282b7b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.585316 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e171021b-1fd4-400f-b059-676d282b7b87 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:32.586513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e171021b-1fd4-400f-b059-676d282b7b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:32.593194 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c02170-6dfa-48aa-a235-c39e60b18bc3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:32.627733 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a63ae-030c-4299-b371-4431c27d3454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:32.628411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a63ae-030c-4299-b371-4431c27d3454 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.628688 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a63ae-030c-4299-b371-4431c27d3454 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.628960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a63ae-030c-4299-b371-4431c27d3454 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.629728 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a63ae-030c-4299-b371-4431c27d3454 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:32.630035 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a63ae-030c-4299-b371-4431c27d3454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:32.631435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a63ae-030c-4299-b371-4431c27d3454 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:32.639435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 269/508] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:32.640249 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a63ae-030c-4299-b371-4431c27d3454 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.651287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:32.657131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 240/509] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:32.658385 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171021b-1fd4-400f-b059-676d282b7b87 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.746641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a63ae-030c-4299-b371-4431c27d3454 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:32.756885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171021b-1fd4-400f-b059-676d282b7b87 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:32.770327 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:32.782178 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ae22f-02e8-43f6-a4db-8f6067062c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:32.783157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ae22f-02e8-43f6-a4db-8f6067062c9d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.783430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ae22f-02e8-43f6-a4db-8f6067062c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.783745 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ae22f-02e8-43f6-a4db-8f6067062c9d None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.784484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ae22f-02e8-43f6-a4db-8f6067062c9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:32.784794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ae22f-02e8-43f6-a4db-8f6067062c9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:32.786204 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ae22f-02e8-43f6-a4db-8f6067062c9d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:32.792075 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d829c1f-a73a-422b-af97-a5e8c4f5e5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:32.792753 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d829c1f-a73a-422b-af97-a5e8c4f5e5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.793033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d829c1f-a73a-422b-af97-a5e8c4f5e5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.793304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d829c1f-a73a-422b-af97-a5e8c4f5e5e3 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.794136 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d829c1f-a73a-422b-af97-a5e8c4f5e5e3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:32.794443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d829c1f-a73a-422b-af97-a5e8c4f5e5e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:32.795910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d829c1f-a73a-422b-af97-a5e8c4f5e5e3 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:32.798444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 270/510] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:32.799346 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ae22f-02e8-43f6-a4db-8f6067062c9d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.831901 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 241/511] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Jun 27 21:03:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:32.833535 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d829c1f-a73a-422b-af97-a5e8c4f5e5e3 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.891478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:32.896923 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ae22f-02e8-43f6-a4db-8f6067062c9d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:32.930025 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3516028-1a1b-4f07-9f01-cbb3c6485ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:32.930825 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3516028-1a1b-4f07-9f01-cbb3c6485ddb None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.931106 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3516028-1a1b-4f07-9f01-cbb3c6485ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.931378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3516028-1a1b-4f07-9f01-cbb3c6485ddb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.935425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d829c1f-a73a-422b-af97-a5e8c4f5e5e3 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:32.939091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3516028-1a1b-4f07-9f01-cbb3c6485ddb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:32.940621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3516028-1a1b-4f07-9f01-cbb3c6485ddb None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:32.957802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-e3516028-1a1b-4f07-9f01-cbb3c6485ddb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 21:03:32.957802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:32.957802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:32.957802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:32.957802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:32.957802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:32.957802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:32.957802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:32.957802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:32.957802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:32.957802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:32.957802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:32.957802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:32.957802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:32.957802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:32.957802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:32.958913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:32.958913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:32.958913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:32.958913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:32.958913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:32.958913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:32.958913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:32.958913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:32.958913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:32.958913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:32.958913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:32.958913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:32.958913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:32.958913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:32.958913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:32.958913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:32.958913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:32.958913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:32.960007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:32.960007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:32.960007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:32.960007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:32.960007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:32.960007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:32.960007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:32.960007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:32.960007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 21:03:32.960007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:32.960007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 271/512] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:32.960642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3516028-1a1b-4f07-9f01-cbb3c6485ddb None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.974416 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347d5f7-adef-43c4-a0d9-6e8ca16089ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:32.975230 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347d5f7-adef-43c4-a0d9-6e8ca16089ff None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.975510 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347d5f7-adef-43c4-a0d9-6e8ca16089ff None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.975812 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347d5f7-adef-43c4-a0d9-6e8ca16089ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.989508 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b347d5f7-adef-43c4-a0d9-6e8ca16089ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:32.991029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b347d5f7-adef-43c4-a0d9-6e8ca16089ff None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:33.009189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-b347d5f7-adef-43c4-a0d9-6e8ca16089ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:33.009189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:33.009189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:33.009189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:33.009189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:33.009189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:33.009189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:33.009189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:33.009189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:33.009189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:33.009189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:33.009189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:33.009189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:33.009189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:33.009189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:33.010198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:33.010198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:33.010198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:33.010198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:33.010198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:33.010198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:33.010198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:33.010198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:33.010198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:33.010198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:33.010198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:33.010198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:33.010198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:33.010198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:33.010198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:33.010198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:33.010198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:33.011211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:33.011211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:33.011211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:33.011211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:33.011211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 242/513] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:33.011211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347d5f7-adef-43c4-a0d9-6e8ca16089ff None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:33.013576 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:33.069769 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3516028-1a1b-4f07-9f01-cbb3c6485ddb None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.102148 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda805ac-99c8-450d-b2ef-1869e0989682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:33.102942 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda805ac-99c8-450d-b2ef-1869e0989682 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.103221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda805ac-99c8-450d-b2ef-1869e0989682 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.103492 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda805ac-99c8-450d-b2ef-1869e0989682 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.110976 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347d5f7-adef-43c4-a0d9-6e8ca16089ff None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.113372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda805ac-99c8-450d-b2ef-1869e0989682 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:33.114840 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda805ac-99c8-450d-b2ef-1869e0989682 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:33.119768 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:33.132302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-eda805ac-99c8-450d-b2ef-1869e0989682 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:33.132302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:33.132302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:33.132302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:33.132302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:33.132302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:33.132302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:33.132302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:33.132302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:33.132302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:33.132302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:33.132302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:33.132302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:33.132302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:33.132302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:33.133308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:33.133308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:33.133308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:33.133308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:33.133308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:33.133308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:33.133308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:33.133308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:33.133308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:33.133308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:33.133308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:33.133308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:33.133308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:33.133308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:33.133308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:33.133308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:33.133308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:33.134244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:33.134244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:33.134244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:33.134244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:33.134244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 272/514] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:33.134244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda805ac-99c8-450d-b2ef-1869e0989682 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:33.148667 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978a418-cf3f-490c-9c12-c6b2a1c01989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:33.149476 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978a418-cf3f-490c-9c12-c6b2a1c01989 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.149758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978a418-cf3f-490c-9c12-c6b2a1c01989 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.150033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978a418-cf3f-490c-9c12-c6b2a1c01989 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.157230 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978a418-cf3f-490c-9c12-c6b2a1c01989 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:33.159151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978a418-cf3f-490c-9c12-c6b2a1c01989 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:33.174366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-9978a418-cf3f-490c-9c12-c6b2a1c01989 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:33.174366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:33.174366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:33.174366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:33.174366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:33.174366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:33.174366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:33.174366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:33.174366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:33.174366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:33.174366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:33.174366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:33.174366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:33.174366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:33.174366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:33.175849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:33.175849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:33.175849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:33.175849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:33.175849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:33.175849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:33.175849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:33.175849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:33.175849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:33.175849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:33.175849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:33.175849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:33.175849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:33.175849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:33.175849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:33.175849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:33.175849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:33.176721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:33.176721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:33.176721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:33.176721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:33.176721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:33.176721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:33.176721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:33.176721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:33.176721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 243/515] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:33.176721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978a418-cf3f-490c-9c12-c6b2a1c01989 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:33.231079 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:33.237662 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda805ac-99c8-450d-b2ef-1869e0989682 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.271062 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed2f59-f584-44bb-be5b-6d2e3f700a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:33.271747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed2f59-f584-44bb-be5b-6d2e3f700a05 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.272034 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed2f59-f584-44bb-be5b-6d2e3f700a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.272310 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed2f59-f584-44bb-be5b-6d2e3f700a05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.277732 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978a418-cf3f-490c-9c12-c6b2a1c01989 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.313758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91572c48-7b36-429c-80b2-23119f9188b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:33.314521 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91572c48-7b36-429c-80b2-23119f9188b0 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.314804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91572c48-7b36-429c-80b2-23119f9188b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.315064 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91572c48-7b36-429c-80b2-23119f9188b0 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.315963 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91572c48-7b36-429c-80b2-23119f9188b0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:33.316256 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91572c48-7b36-429c-80b2-23119f9188b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:33.317738 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91572c48-7b36-429c-80b2-23119f9188b0 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:33.336547 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 244/516] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Jun 27 21:03:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:33.337319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91572c48-7b36-429c-80b2-23119f9188b0 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:33.342795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed2f59-f584-44bb-be5b-6d2e3f700a05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:33.344314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed2f59-f584-44bb-be5b-6d2e3f700a05 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:33.363626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:33.438106 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:33.439715 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91572c48-7b36-429c-80b2-23119f9188b0 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.444989 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 273/517] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:33.445811 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed2f59-f584-44bb-be5b-6d2e3f700a05 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:33.477343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a7e88-a317-43fd-a4dc-44b79fc29a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:33.478017 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a7e88-a317-43fd-a4dc-44b79fc29a04 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.478303 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a7e88-a317-43fd-a4dc-44b79fc29a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.478581 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a7e88-a317-43fd-a4dc-44b79fc29a04 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.479307 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a7e88-a317-43fd-a4dc-44b79fc29a04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:33.479638 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a7e88-a317-43fd-a4dc-44b79fc29a04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:33.481044 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a7e88-a317-43fd-a4dc-44b79fc29a04 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:33.492636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 245/518] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:33.493421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a7e88-a317-43fd-a4dc-44b79fc29a04 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:33.551767 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed2f59-f584-44bb-be5b-6d2e3f700a05 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.578681 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:33.584735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ca3b1-2f0c-45f7-b1de-03ea0dff430c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:33.585398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ca3b1-2f0c-45f7-b1de-03ea0dff430c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.585678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ca3b1-2f0c-45f7-b1de-03ea0dff430c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.585948 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ca3b1-2f0c-45f7-b1de-03ea0dff430c None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.586669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ca3b1-2f0c-45f7-b1de-03ea0dff430c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:33.586976 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ca3b1-2f0c-45f7-b1de-03ea0dff430c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:33.588405 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ca3b1-2f0c-45f7-b1de-03ea0dff430c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:33.594304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a7e88-a317-43fd-a4dc-44b79fc29a04 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.599573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 274/519] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:33.600380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ca3b1-2f0c-45f7-b1de-03ea0dff430c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:33.631994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8abc24-1dd7-4238-b6c8-a0d4dcc220d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:33.632665 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8abc24-1dd7-4238-b6c8-a0d4dcc220d0 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.632955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8abc24-1dd7-4238-b6c8-a0d4dcc220d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.633240 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8abc24-1dd7-4238-b6c8-a0d4dcc220d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.634069 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8abc24-1dd7-4238-b6c8-a0d4dcc220d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:33.634384 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8abc24-1dd7-4238-b6c8-a0d4dcc220d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:33.635928 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8abc24-1dd7-4238-b6c8-a0d4dcc220d0 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:33.653335 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:33.653833 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 246/520] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:33.654667 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8abc24-1dd7-4238-b6c8-a0d4dcc220d0 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:33.699027 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ca3b1-2f0c-45f7-b1de-03ea0dff430c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.732061 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b7971-3b6c-4a7d-9f22-b6bcbbda70f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:33.732714 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b7971-3b6c-4a7d-9f22-b6bcbbda70f5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.732991 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b7971-3b6c-4a7d-9f22-b6bcbbda70f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.733259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b7971-3b6c-4a7d-9f22-b6bcbbda70f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.757472 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8abc24-1dd7-4238-b6c8-a0d4dcc220d0 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.790910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a91c33-cde5-451a-8c64-f40b550e2559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:33.791739 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a91c33-cde5-451a-8c64-f40b550e2559 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.792027 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a91c33-cde5-451a-8c64-f40b550e2559 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.792304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a91c33-cde5-451a-8c64-f40b550e2559 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.800195 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b7971-3b6c-4a7d-9f22-b6bcbbda70f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:33.801672 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b7971-3b6c-4a7d-9f22-b6bcbbda70f5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:33.802628 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a91c33-cde5-451a-8c64-f40b550e2559 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:33.804089 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a91c33-cde5-451a-8c64-f40b550e2559 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:33.810839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:33.822503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-d0a91c33-cde5-451a-8c64-f40b550e2559 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:33.822503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:33.822503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:33.822503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:33.822503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:33.822503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:33.822503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:33.822503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:33.822503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:33.822503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:33.822503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:33.822503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:33.822503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:33.822503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:33.822503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:33.823340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:33.823340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:33.823340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:33.823340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:33.823340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:33.823340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:33.823340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:33.823340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:33.823340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:33.823340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:33.823340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:33.823340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:33.823340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:33.823340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:33.823340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:33.823340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:33.823340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:33.824480 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:33.824480 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:33.824480 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:33.824480 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:33.824480 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 247/521] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:33.824901 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a91c33-cde5-451a-8c64-f40b550e2559 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:33.858399 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:33.902989 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 275/522] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:33.903644 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b7971-3b6c-4a7d-9f22-b6bcbbda70f5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:33.930093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a91c33-cde5-451a-8c64-f40b550e2559 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.963520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3513ec-8587-47cf-aca7-0aa7f57833d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:33.964355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3513ec-8587-47cf-aca7-0aa7f57833d7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.964634 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3513ec-8587-47cf-aca7-0aa7f57833d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.964906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3513ec-8587-47cf-aca7-0aa7f57833d7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.972016 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3513ec-8587-47cf-aca7-0aa7f57833d7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:33.973490 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3513ec-8587-47cf-aca7-0aa7f57833d7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:33.992840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-2c3513ec-8587-47cf-aca7-0aa7f57833d7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 21:03:33.992840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:33.992840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:33.992840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:33.992840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:33.992840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:33.992840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:33.992840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:33.992840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:33.992840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:33.992840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:33.992840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:33.992840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:33.992840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:33.992840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:33.992840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:33.993703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:33.993703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:33.993703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:33.993703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:33.993703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:33.993703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:33.993703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:33.993703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:33.993703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:33.993703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:33.993703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:33.993703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:33.993703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:33.993703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:33.993703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:33.993703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:33.993703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:33.993703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:33.994818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:33.994818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:33.994818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:33.994818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:33.994818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:33.994818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:33.994818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:33.994818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:33.994818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 21:03:33.994818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:33.994818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 248/523] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:33.994818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3513ec-8587-47cf-aca7-0aa7f57833d7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:34.007030 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b7971-3b6c-4a7d-9f22-b6bcbbda70f5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:34.042054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8c4ee-1332-4514-9330-66f3722b5473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:34.042721 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8c4ee-1332-4514-9330-66f3722b5473 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:34.043005 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8c4ee-1332-4514-9330-66f3722b5473 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:34.043279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8c4ee-1332-4514-9330-66f3722b5473 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:34.044021 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8c4ee-1332-4514-9330-66f3722b5473 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:34.045472 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8c4ee-1332-4514-9330-66f3722b5473 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:34.059955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:34.065491 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 276/524] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Jun 27 21:03:33 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:34.066303 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8c4ee-1332-4514-9330-66f3722b5473 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:34.082294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:34.114701 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3513ec-8587-47cf-aca7-0aa7f57833d7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:34.156262 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3412b3-0585-409e-8d4e-c0fd32b1ea39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:34.156960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3412b3-0585-409e-8d4e-c0fd32b1ea39 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:34.157438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3412b3-0585-409e-8d4e-c0fd32b1ea39 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:34.157544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3412b3-0585-409e-8d4e-c0fd32b1ea39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:34.158740 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3412b3-0585-409e-8d4e-c0fd32b1ea39 None admin] RBAC: Adding query filter params (role.id=fb120deba4284112b12de1a251b7da76, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=8d7fe01ef64f46f6a0e77d5945219b33) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:34.159070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3412b3-0585-409e-8d4e-c0fd32b1ea39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:34.160516 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3412b3-0585-409e-8d4e-c0fd32b1ea39 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:34.179657 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 249/525] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:33 2022] GET /identity/v3/role_assignments?user.id=8d7fe01ef64f46f6a0e77d5945219b33&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=fb120deba4284112b12de1a251b7da76 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:34.179657 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3412b3-0585-409e-8d4e-c0fd32b1ea39 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:34.215448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8c4ee-1332-4514-9330-66f3722b5473 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:34.270273 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c639fc9-07c8-4a31-af1a-31a7286dbb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:34.270961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c639fc9-07c8-4a31-af1a-31a7286dbb23 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:34.271241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c639fc9-07c8-4a31-af1a-31a7286dbb23 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:34.271516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c639fc9-07c8-4a31-af1a-31a7286dbb23 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:34.272353 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c639fc9-07c8-4a31-af1a-31a7286dbb23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:34.272661 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c639fc9-07c8-4a31-af1a-31a7286dbb23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:34.274095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c639fc9-07c8-4a31-af1a-31a7286dbb23 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:34.292336 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 277/526] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:34.293263 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c639fc9-07c8-4a31-af1a-31a7286dbb23 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:34.317561 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3412b3-0585-409e-8d4e-c0fd32b1ea39 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:34.347803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:34.370313 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc81cc-457f-4dfc-9590-daf1d7e7fdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:34.371710 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc81cc-457f-4dfc-9590-daf1d7e7fdb4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:34.372126 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc81cc-457f-4dfc-9590-daf1d7e7fdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:34.372407 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc81cc-457f-4dfc-9590-daf1d7e7fdb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:34.379893 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:34.389335 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc81cc-457f-4dfc-9590-daf1d7e7fdb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:34.392253 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc81cc-457f-4dfc-9590-daf1d7e7fdb4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:34.414285 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c639fc9-07c8-4a31-af1a-31a7286dbb23 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:34.417631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-1cdc81cc-457f-4dfc-9590-daf1d7e7fdb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:34.417631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:34.417631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:34.417631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:34.417631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:34.417631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:34.417631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:34.417631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:34.417631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:34.417631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:34.417631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:34.417631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:34.417631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:34.417631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:34.417631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:34.418478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:34.418478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:34.418478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:34.418478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:34.418478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:34.418478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:34.418478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:34.418478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:34.418478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:34.418478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:34.418478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:34.418478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:34.418478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:34.418478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:34.418478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:34.418478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:34.418478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:34.419761 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:34.419761 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:34.419761 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:34.419761 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:34.419761 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 250/527] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:34.420162 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc81cc-457f-4dfc-9590-daf1d7e7fdb4 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:34.459987 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada6499-bf2a-4e17-bd8b-15b7260f7c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:34.460671 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada6499-bf2a-4e17-bd8b-15b7260f7c93 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:34.460959 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada6499-bf2a-4e17-bd8b-15b7260f7c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:34.461233 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada6499-bf2a-4e17-bd8b-15b7260f7c93 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:34.462079 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada6499-bf2a-4e17-bd8b-15b7260f7c93 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:34.462391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada6499-bf2a-4e17-bd8b-15b7260f7c93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:34.464443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada6499-bf2a-4e17-bd8b-15b7260f7c93 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:34.489586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 278/528] 10.222.0.27 () {56 vars in 1139 bytes} [Mon Jun 27 21:03:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:34.490707 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada6499-bf2a-4e17-bd8b-15b7260f7c93 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:34.527922 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc81cc-457f-4dfc-9590-daf1d7e7fdb4 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:34.564657 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c33a6c-0ae3-4e1b-8d52-b97af3500e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:34.565731 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c33a6c-0ae3-4e1b-8d52-b97af3500e5c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:34.566016 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c33a6c-0ae3-4e1b-8d52-b97af3500e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:34.566287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c33a6c-0ae3-4e1b-8d52-b97af3500e5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:34.575648 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c33a6c-0ae3-4e1b-8d52-b97af3500e5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:34.577059 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c33a6c-0ae3-4e1b-8d52-b97af3500e5c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:34.600641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-95c33a6c-0ae3-4e1b-8d52-b97af3500e5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:34.600641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:34.600641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:34.600641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:34.600641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:34.600641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:34.600641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:34.600641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:34.600641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:34.600641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:34.600641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:34.600641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:34.600641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:34.600641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:34.600641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:34.602595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:34.602595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:34.602595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:34.602595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:34.602595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:34.602595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:34.602595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:34.602595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:34.602595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:34.602595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:34.602595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:34.602595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:34.602595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:34.602595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:34.602595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:34.602595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:34.602595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:34.604558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:34.604558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:34.604558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:34.604558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:34.604558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:34.604558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:34.604558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:34.604558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:34.611646 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 251/529] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:34.611646 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c33a6c-0ae3-4e1b-8d52-b97af3500e5c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:34.624579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada6499-bf2a-4e17-bd8b-15b7260f7c93 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:34.655493 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:34.662656 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:34.664639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b27fc-2f47-4137-99a2-021839656bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:34.665388 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b27fc-2f47-4137-99a2-021839656bab None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:34.665751 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b27fc-2f47-4137-99a2-021839656bab None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:34.666027 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b27fc-2f47-4137-99a2-021839656bab None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:34.666751 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b27fc-2f47-4137-99a2-021839656bab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:34.667057 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b27fc-2f47-4137-99a2-021839656bab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:34.668554 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b27fc-2f47-4137-99a2-021839656bab None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:34.682832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 279/530] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:34.683877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b27fc-2f47-4137-99a2-021839656bab None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:34.739617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c33a6c-0ae3-4e1b-8d52-b97af3500e5c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:34.775276 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f8bf0-d2ab-4619-abf9-ecebf65c4695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:34.775990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f8bf0-d2ab-4619-abf9-ecebf65c4695 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:34.776270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f8bf0-d2ab-4619-abf9-ecebf65c4695 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:34.776543 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f8bf0-d2ab-4619-abf9-ecebf65c4695 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:34.809112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b27fc-2f47-4137-99a2-021839656bab None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:34.852084 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86071709-9200-4f52-8cf8-af3ffacfad21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:34.852970 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86071709-9200-4f52-8cf8-af3ffacfad21 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:34.853256 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86071709-9200-4f52-8cf8-af3ffacfad21 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:34.853533 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86071709-9200-4f52-8cf8-af3ffacfad21 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:34.854334 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86071709-9200-4f52-8cf8-af3ffacfad21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:34.854642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86071709-9200-4f52-8cf8-af3ffacfad21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:34.856801 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86071709-9200-4f52-8cf8-af3ffacfad21 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:34.858195 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f8bf0-d2ab-4619-abf9-ecebf65c4695 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:34.859858 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f8bf0-d2ab-4619-abf9-ecebf65c4695 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:34.878760 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 280/531] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:34.879638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86071709-9200-4f52-8cf8-af3ffacfad21 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:34.903488 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:34.941973 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:34.993825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 252/532] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:34.999250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f8bf0-d2ab-4619-abf9-ecebf65c4695 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:35.013779 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86071709-9200-4f52-8cf8-af3ffacfad21 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:35.058102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d736676-a802-496d-9552-77c3da5333b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:35.059205 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d736676-a802-496d-9552-77c3da5333b9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:35.060209 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d736676-a802-496d-9552-77c3da5333b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:35.060687 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d736676-a802-496d-9552-77c3da5333b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:35.070423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d736676-a802-496d-9552-77c3da5333b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:35.072380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d736676-a802-496d-9552-77c3da5333b9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:35.091332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4d736676-a802-496d-9552-77c3da5333b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:35.091332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:35.091332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:35.091332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:35.091332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:35.091332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:35.091332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:35.091332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:35.091332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:35.091332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:35.091332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:35.091332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:35.091332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:35.091332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:35.091332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:35.092216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:35.092216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:35.092216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:35.092216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:35.092216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:35.092216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:35.092216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:35.092216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:35.092216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:35.092216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:35.092216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:35.092216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:35.092216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:35.092216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:35.092216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:35.092216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:35.092216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:35.093040 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:35.093040 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:35.093040 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:35.093040 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:35.093040 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:35.093040 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:35.093040 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:35.093040 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:35.093040 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 281/533] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:35.093487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d736676-a802-496d-9552-77c3da5333b9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:35.115270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f8bf0-d2ab-4619-abf9-ecebf65c4695 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:35.153264 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851fb77a-700f-4338-a21c-73b966b8bd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:35.154152 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851fb77a-700f-4338-a21c-73b966b8bd5d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:35.154607 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851fb77a-700f-4338-a21c-73b966b8bd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:35.155060 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851fb77a-700f-4338-a21c-73b966b8bd5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:35.156205 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851fb77a-700f-4338-a21c-73b966b8bd5d None admin] RBAC: Adding query filter params (role.id=fb120deba4284112b12de1a251b7da76, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=b43db1fd0b494a3f8588b498b585f247) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:35.156680 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851fb77a-700f-4338-a21c-73b966b8bd5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:35.158643 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851fb77a-700f-4338-a21c-73b966b8bd5d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:35.160810 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:35.166151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 253/534] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:34 2022] GET /identity/v3/role_assignments?user.id=b43db1fd0b494a3f8588b498b585f247&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=fb120deba4284112b12de1a251b7da76 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:35.167121 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851fb77a-700f-4338-a21c-73b966b8bd5d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:35.218082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d736676-a802-496d-9552-77c3da5333b9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:35.236161 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:35.286636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851fb77a-700f-4338-a21c-73b966b8bd5d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:35.289442 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c1265-0f7a-4e27-b6aa-ab40b462c577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:35.290243 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c1265-0f7a-4e27-b6aa-ab40b462c577 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:35.290526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c1265-0f7a-4e27-b6aa-ab40b462c577 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:35.290798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c1265-0f7a-4e27-b6aa-ab40b462c577 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:35.312873 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c1265-0f7a-4e27-b6aa-ab40b462c577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:35.314100 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c1265-0f7a-4e27-b6aa-ab40b462c577 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:35.329773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8fd6b-9cfb-46a7-a26d-72b1ed1f4b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:35.331686 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8fd6b-9cfb-46a7-a26d-72b1ed1f4b5d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:35.332224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8fd6b-9cfb-46a7-a26d-72b1ed1f4b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:35.332697 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8fd6b-9cfb-46a7-a26d-72b1ed1f4b5d None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:35.333858 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8fd6b-9cfb-46a7-a26d-72b1ed1f4b5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:35.334366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8fd6b-9cfb-46a7-a26d-72b1ed1f4b5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:35.336195 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8fd6b-9cfb-46a7-a26d-72b1ed1f4b5d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:35.350577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 254/535] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:35.352876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-9e5c1265-0f7a-4e27-b6aa-ab40b462c577 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:35.352876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:35.352876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:35.352876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:35.352876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:35.352876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:35.352876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:35.352876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:35.352876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:35.352876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:35.352876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:35.352876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:35.352876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:35.352876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:35.352876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:35.353693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:35.353693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:35.353693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:35.353693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:35.353693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:35.353693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:35.353693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:35.353693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:35.353693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:35.353693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:35.353693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:35.353693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:35.353693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:35.353693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:35.353693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:35.353693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:35.353693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:35.354518 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:35.354518 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:35.354518 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:35.354518 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:35.356213 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8fd6b-9cfb-46a7-a26d-72b1ed1f4b5d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:35.358250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 282/536] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:35.363910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c1265-0f7a-4e27-b6aa-ab40b462c577 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:35.429568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:35.478301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8fd6b-9cfb-46a7-a26d-72b1ed1f4b5d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:35.517260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ea0c5-3d51-4f18-aa3c-dd2961297bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:35.518357 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ea0c5-3d51-4f18-aa3c-dd2961297bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:35.518833 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ea0c5-3d51-4f18-aa3c-dd2961297bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:35.519349 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ea0c5-3d51-4f18-aa3c-dd2961297bd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:35.520561 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ea0c5-3d51-4f18-aa3c-dd2961297bd8 None admin] RBAC: Adding query filter params (role.id=fb120deba4284112b12de1a251b7da76, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=16fda36445d9415aae4df74f31af0601) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:35.521189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ea0c5-3d51-4f18-aa3c-dd2961297bd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:35.522826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ea0c5-3d51-4f18-aa3c-dd2961297bd8 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:35.526579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c1265-0f7a-4e27-b6aa-ab40b462c577 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:35.529724 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 255/537] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:35 2022] GET /identity/v3/role_assignments?user.id=16fda36445d9415aae4df74f31af0601&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=fb120deba4284112b12de1a251b7da76 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:35.574841 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa7e15-6c2f-43ef-88bf-2c5e19b6231b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:35.575541 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa7e15-6c2f-43ef-88bf-2c5e19b6231b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:35.575842 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa7e15-6c2f-43ef-88bf-2c5e19b6231b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:35.576115 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa7e15-6c2f-43ef-88bf-2c5e19b6231b None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:35.576851 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa7e15-6c2f-43ef-88bf-2c5e19b6231b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:35.577156 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa7e15-6c2f-43ef-88bf-2c5e19b6231b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:35.578598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa7e15-6c2f-43ef-88bf-2c5e19b6231b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:35.591822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 283/538] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:35.596958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ea0c5-3d51-4f18-aa3c-dd2961297bd8 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:35.710862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ea0c5-3d51-4f18-aa3c-dd2961297bd8 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:35.737109 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:35.750014 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55115e-196b-48b0-a910-6fe3373bc3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:35.751446 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55115e-196b-48b0-a910-6fe3373bc3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:35.752078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55115e-196b-48b0-a910-6fe3373bc3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:35.752550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55115e-196b-48b0-a910-6fe3373bc3d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:35.760414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55115e-196b-48b0-a910-6fe3373bc3d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:35.762194 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55115e-196b-48b0-a910-6fe3373bc3d5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:35.777550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-ba55115e-196b-48b0-a910-6fe3373bc3d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:35.777550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:35.777550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:35.777550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:35.777550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:35.777550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:35.777550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:35.777550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:35.777550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:35.777550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:35.777550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:35.777550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:35.777550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:35.777550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:35.777550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:35.778443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:35.778443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:35.778443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:35.778443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:35.778443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:35.778443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:35.778443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:35.778443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:35.778443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:35.778443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:35.778443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:35.778443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:35.778443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:35.778443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:35.778443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:35.778443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:35.778443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:35.779295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:35.779295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:35.779295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:35.779295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:35.779295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:35.779295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:35.779295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:35.779295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:35.781883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 256/539] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:35.788963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa7e15-6c2f-43ef-88bf-2c5e19b6231b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:35.838734 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:35.925170 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa7e15-6c2f-43ef-88bf-2c5e19b6231b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:35.974305 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa7199-3e4f-4405-82f3-4cf130fc271f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:35.977491 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa7199-3e4f-4405-82f3-4cf130fc271f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:35.977491 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa7199-3e4f-4405-82f3-4cf130fc271f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:35.977491 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa7199-3e4f-4405-82f3-4cf130fc271f None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:35.977491 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa7199-3e4f-4405-82f3-4cf130fc271f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:35.977491 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa7199-3e4f-4405-82f3-4cf130fc271f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:35.978015 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa7199-3e4f-4405-82f3-4cf130fc271f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:35.996387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 284/540] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:36.002441 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55115e-196b-48b0-a910-6fe3373bc3d5 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:36.131214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55115e-196b-48b0-a910-6fe3373bc3d5 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:36.166874 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86264e65-e459-4c44-b812-c871e1bd10d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:36.167907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86264e65-e459-4c44-b812-c871e1bd10d5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.168356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86264e65-e459-4c44-b812-c871e1bd10d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.168797 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86264e65-e459-4c44-b812-c871e1bd10d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.169792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86264e65-e459-4c44-b812-c871e1bd10d5 None admin] RBAC: Adding query filter params (role.id=fb120deba4284112b12de1a251b7da76, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=9c7ce29db6b14176a4c9be436bcddc6e) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:36.170272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86264e65-e459-4c44-b812-c871e1bd10d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:36.171912 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86264e65-e459-4c44-b812-c871e1bd10d5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:36.178376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 257/541] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:36 2022] GET /identity/v3/role_assignments?user.id=9c7ce29db6b14176a4c9be436bcddc6e&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=fb120deba4284112b12de1a251b7da76 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:36.308950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe1b22-7110-49f5-95fa-e8fdf193f964 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.309879 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe1b22-7110-49f5-95fa-e8fdf193f964 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.310330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe1b22-7110-49f5-95fa-e8fdf193f964 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.312261 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 285/542] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:36.317585 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd85eee-2219-4b25-9511-c6ec59c04efb None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.318438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd85eee-2219-4b25-9511-c6ec59c04efb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.318881 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd85eee-2219-4b25-9511-c6ec59c04efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.333178 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c545387-f0de-4922-8c9a-5ca2d9970366 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.333773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c545387-f0de-4922-8c9a-5ca2d9970366 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.334216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c545387-f0de-4922-8c9a-5ca2d9970366 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.335809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 286/543] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:36.341130 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98425699-3b6b-41a7-a2fa-f56c9d29e32d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.341626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98425699-3b6b-41a7-a2fa-f56c9d29e32d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.342092 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98425699-3b6b-41a7-a2fa-f56c9d29e32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.355294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-8fd85eee-2219-4b25-9511-c6ec59c04efb None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:36.356595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8fd85eee-2219-4b25-9511-c6ec59c04efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:36.367834 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-98425699-3b6b-41a7-a2fa-f56c9d29e32d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:36.369492 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-98425699-3b6b-41a7-a2fa-f56c9d29e32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:36.426683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:36.443388 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 287/544] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:36.470644 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98425699-3b6b-41a7-a2fa-f56c9d29e32d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:36.475453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 258/545] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:36.485939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:36.511120 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd85eee-2219-4b25-9511-c6ec59c04efb None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:36.627787 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98425699-3b6b-41a7-a2fa-f56c9d29e32d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:36.664674 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c83f6a-4e9c-4517-95d4-a631d3b19b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:36.665491 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c83f6a-4e9c-4517-95d4-a631d3b19b4d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.665778 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c83f6a-4e9c-4517-95d4-a631d3b19b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.666061 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c83f6a-4e9c-4517-95d4-a631d3b19b4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.670405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd85eee-2219-4b25-9511-c6ec59c04efb None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:36.688597 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c83f6a-4e9c-4517-95d4-a631d3b19b4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:36.702135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-28c83f6a-4e9c-4517-95d4-a631d3b19b4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:36.702135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:36.702135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:36.702135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:36.702135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:36.702135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:36.702135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:36.702135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:36.702135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:36.702135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:36.702135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:36.702135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:36.702135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:36.702135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:36.702135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:36.702135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:36.703075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:36.703075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:36.703075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:36.703075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:36.703075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:36.703075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:36.703075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:36.703075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:36.703075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:36.703075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:36.703075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:36.703075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:36.703075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:36.703075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:36.703075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:36.703075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:36.703878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:36.703878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:36.703878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:36.703878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:36.703878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:36.703878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:36.703878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:36.703878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:36.703878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:36.703878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:36.703878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:36.703878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:36.703878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 288/546] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:36.710572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c83f6a-4e9c-4517-95d4-a631d3b19b4d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:36.724155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e42596-ddc0-478e-9d88-49fe1686e1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:36.726050 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e42596-ddc0-478e-9d88-49fe1686e1da None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.726341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e42596-ddc0-478e-9d88-49fe1686e1da None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.726620 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e42596-ddc0-478e-9d88-49fe1686e1da None admin] PATH_INFO: `/v3/roles/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.746164 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e42596-ddc0-478e-9d88-49fe1686e1da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:36.755759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-99e42596-ddc0-478e-9d88-49fe1686e1da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:36.755759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:36.755759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:36.755759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:36.755759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:36.755759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:36.755759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:36.755759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:36.755759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:36.755759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:36.755759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:36.755759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:36.755759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:36.755759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:36.755759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:36.755759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:36.756637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:36.756637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:36.756637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:36.756637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:36.756637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:36.756637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:36.756637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:36.756637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:36.756637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:36.756637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:36.756637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:36.756637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:36.756637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:36.756637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:36.756637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:36.756637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:36.757444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:36.757444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:36.757444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:36.757444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:36.757444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:36.757444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:36.757444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:36.757444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:36.757444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:36.757444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:36.757444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:36.757444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:36.758038 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 259/547] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:36.763634 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e42596-ddc0-478e-9d88-49fe1686e1da None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:36.805470 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:36.864779 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:36.893305 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e42596-ddc0-478e-9d88-49fe1686e1da None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:36.927727 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c83f6a-4e9c-4517-95d4-a631d3b19b4d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:36.962742 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53e0c4-d6e9-4a18-aa13-31e07f53541f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:36.963846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53e0c4-d6e9-4a18-aa13-31e07f53541f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.964476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53e0c4-d6e9-4a18-aa13-31e07f53541f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.964943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53e0c4-d6e9-4a18-aa13-31e07f53541f None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.965871 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53e0c4-d6e9-4a18-aa13-31e07f53541f None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:36.966384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53e0c4-d6e9-4a18-aa13-31e07f53541f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:36.968116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53e0c4-d6e9-4a18-aa13-31e07f53541f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:36.977172 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 289/548] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:36.985926 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53e0c4-d6e9-4a18-aa13-31e07f53541f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:36.992682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1607835-bb0c-45a7-9190-abc6e09f008f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:36.993738 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1607835-bb0c-45a7-9190-abc6e09f008f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.993738 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1607835-bb0c-45a7-9190-abc6e09f008f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.993893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1607835-bb0c-45a7-9190-abc6e09f008f None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.997866 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1607835-bb0c-45a7-9190-abc6e09f008f None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:36.997866 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1607835-bb0c-45a7-9190-abc6e09f008f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:36.997866 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1607835-bb0c-45a7-9190-abc6e09f008f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:37.007280 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 260/549] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:37.012512 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1607835-bb0c-45a7-9190-abc6e09f008f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:37.158297 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53e0c4-d6e9-4a18-aa13-31e07f53541f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:37.188412 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:37.197721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1607835-bb0c-45a7-9190-abc6e09f008f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:37.198335 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33cdca-d9b2-434e-8df1-fe5da900d6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:37.199639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33cdca-d9b2-434e-8df1-fe5da900d6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:37.200113 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33cdca-d9b2-434e-8df1-fe5da900d6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:37.200595 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33cdca-d9b2-434e-8df1-fe5da900d6d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:37.202826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:37.214163 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33cdca-d9b2-434e-8df1-fe5da900d6d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:37.215951 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33cdca-d9b2-434e-8df1-fe5da900d6d3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:37.235877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-ab33cdca-d9b2-434e-8df1-fe5da900d6d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:37.235877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:37.235877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:37.235877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:37.235877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:37.235877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:37.235877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:37.235877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:37.235877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:37.235877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:37.235877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:37.235877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:37.235877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:37.235877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:37.235877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:37.237321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:37.237321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:37.237321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:37.237321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:37.237321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:37.237321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:37.237321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:37.237321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:37.237321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:37.237321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:37.237321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:37.237321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:37.237321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:37.237321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:37.237321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:37.237321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:37.237321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:37.238315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:37.238315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:37.238315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:37.238315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:37.241029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 290/550] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:37.249061 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33cdca-d9b2-434e-8df1-fe5da900d6d3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:37.279869 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae42ea-8b66-4ab2-8d65-26acc7c55fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:37.282200 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae42ea-8b66-4ab2-8d65-26acc7c55fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:37.282688 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae42ea-8b66-4ab2-8d65-26acc7c55fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:37.284289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae42ea-8b66-4ab2-8d65-26acc7c55fa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:37.304600 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae42ea-8b66-4ab2-8d65-26acc7c55fa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:37.307375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae42ea-8b66-4ab2-8d65-26acc7c55fa1 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:37.335520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-6bae42ea-8b66-4ab2-8d65-26acc7c55fa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:37.335520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:37.335520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:37.335520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:37.335520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:37.335520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:37.335520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:37.335520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:37.335520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:37.335520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:37.335520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:37.335520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:37.335520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:37.335520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:37.335520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:37.336862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:37.336862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:37.336862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:37.336862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:37.336862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:37.336862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:37.336862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:37.336862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:37.336862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:37.336862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:37.336862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:37.336862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:37.336862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:37.336862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:37.336862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:37.336862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:37.336862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:37.337729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:37.337729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:37.337729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:37.337729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:37.350080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 261/551] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:37.352546 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae42ea-8b66-4ab2-8d65-26acc7c55fa1 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:37.418048 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33cdca-d9b2-434e-8df1-fe5da900d6d3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:37.451569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fec7f5-d609-45ed-87ab-2f3c0f5e9fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:37.452331 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fec7f5-d609-45ed-87ab-2f3c0f5e9fbb None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:37.452612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fec7f5-d609-45ed-87ab-2f3c0f5e9fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:37.452883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fec7f5-d609-45ed-87ab-2f3c0f5e9fbb None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:37.453716 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fec7f5-d609-45ed-87ab-2f3c0f5e9fbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:37.453954 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fec7f5-d609-45ed-87ab-2f3c0f5e9fbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:37.455441 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fec7f5-d609-45ed-87ab-2f3c0f5e9fbb None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:37.463110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae42ea-8b66-4ab2-8d65-26acc7c55fa1 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:37.471871 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 291/552] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:37.475647 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fec7f5-d609-45ed-87ab-2f3c0f5e9fbb None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:37.503179 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b123a2-0b4c-447d-932c-dbd19d35ae65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:37.504312 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b123a2-0b4c-447d-932c-dbd19d35ae65 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:37.504787 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b123a2-0b4c-447d-932c-dbd19d35ae65 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:37.505249 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b123a2-0b4c-447d-932c-dbd19d35ae65 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:37.506185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b123a2-0b4c-447d-932c-dbd19d35ae65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:37.506690 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b123a2-0b4c-447d-932c-dbd19d35ae65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:37.508432 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b123a2-0b4c-447d-932c-dbd19d35ae65 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:37.525746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 262/553] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:37.527236 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b123a2-0b4c-447d-932c-dbd19d35ae65 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:37.529974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:37.550892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:37.607465 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fec7f5-d609-45ed-87ab-2f3c0f5e9fbb None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:37.680488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b838455-f63e-485f-8be9-00ea52c74d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:37.681238 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b838455-f63e-485f-8be9-00ea52c74d6e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:37.681547 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b838455-f63e-485f-8be9-00ea52c74d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:37.681812 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b838455-f63e-485f-8be9-00ea52c74d6e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:37.681996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b123a2-0b4c-447d-932c-dbd19d35ae65 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:37.690090 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b838455-f63e-485f-8be9-00ea52c74d6e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:37.691574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b838455-f63e-485f-8be9-00ea52c74d6e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:37.706735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-7b838455-f63e-485f-8be9-00ea52c74d6e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:37.706735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:37.706735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:37.706735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:37.706735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:37.706735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:37.706735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:37.706735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:37.706735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:37.706735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:37.706735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:37.706735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:37.706735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:37.706735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:37.706735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:37.706735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:37.707789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:37.707789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:37.707789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:37.707789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:37.707789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:37.707789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:37.707789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:37.707789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:37.707789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:37.707789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:37.707789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:37.707789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:37.707789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:37.707789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:37.707789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:37.707789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:37.707789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:37.707789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:37.708986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:37.708986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:37.708986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:37.708986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:37.708986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:37.708986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:37.708986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:37.708986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:37.708986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:37.708986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:37.708986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 292/554] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Jun 27 21:03:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:37.721235 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b838455-f63e-485f-8be9-00ea52c74d6e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:37.723171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8bb36-03da-432a-b01f-b661c78e5c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:37.724024 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8bb36-03da-432a-b01f-b661c78e5c62 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:37.724305 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8bb36-03da-432a-b01f-b661c78e5c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:37.724577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8bb36-03da-432a-b01f-b661c78e5c62 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:37.732186 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8bb36-03da-432a-b01f-b661c78e5c62 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:37.733670 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8bb36-03da-432a-b01f-b661c78e5c62 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:37.749224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-33b8bb36-03da-432a-b01f-b661c78e5c62 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:03:37.749224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:37.749224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:37.749224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:37.749224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:37.749224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:37.749224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:37.749224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:37.749224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:37.749224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:37.749224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:37.749224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:37.749224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:37.749224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:37.749224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:37.749224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:37.751431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:37.751431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:37.751431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:37.751431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:37.751431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:37.751431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:37.751431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:37.751431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:37.751431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:37.751431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:37.751431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:37.751431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:37.751431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:37.751431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:37.751431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:37.751431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:37.751431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:37.751431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:37.754214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:37.754214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:37.754214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:37.754214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:37.754214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:37.754214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:37.754214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:37.754214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:37.754214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:03:37.754214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:37.754214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 263/555] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:37.761163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8bb36-03da-432a-b01f-b661c78e5c62 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:37.835126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b838455-f63e-485f-8be9-00ea52c74d6e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:37.843979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:37.848148 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:37.880232 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8bb36-03da-432a-b01f-b661c78e5c62 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:37.883597 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f5be9-6f89-4dee-9b73-48a311d05043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:37.884584 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f5be9-6f89-4dee-9b73-48a311d05043 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:37.885067 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f5be9-6f89-4dee-9b73-48a311d05043 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:37.885541 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f5be9-6f89-4dee-9b73-48a311d05043 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:37.886578 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f5be9-6f89-4dee-9b73-48a311d05043 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:37.887086 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f5be9-6f89-4dee-9b73-48a311d05043 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:37.889089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f5be9-6f89-4dee-9b73-48a311d05043 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:37.909514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 293/556] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Jun 27 21:03:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:37.911256 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b28efa-340a-4a99-8bff-af1729b3aa9d None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:37.911793 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b28efa-340a-4a99-8bff-af1729b3aa9d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:37.912271 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b28efa-340a-4a99-8bff-af1729b3aa9d None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:37.913964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 294/557] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:37.914859 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b28efa-340a-4a99-8bff-af1729b3aa9d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:37.923992 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5811e95-6bb8-48fe-9b48-a766f1ae7dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:37.924673 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5811e95-6bb8-48fe-9b48-a766f1ae7dcf None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:37.924953 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5811e95-6bb8-48fe-9b48-a766f1ae7dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:37.925559 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5811e95-6bb8-48fe-9b48-a766f1ae7dcf None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:37.926063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5811e95-6bb8-48fe-9b48-a766f1ae7dcf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:37.926371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5811e95-6bb8-48fe-9b48-a766f1ae7dcf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:37.927866 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5811e95-6bb8-48fe-9b48-a766f1ae7dcf None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:37.946086 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 264/558] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Jun 27 21:03:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:37.947387 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4236080-10a1-4e1a-aea4-1104e29d2c18 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:37.947707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4236080-10a1-4e1a-aea4-1104e29d2c18 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:37.948234 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4236080-10a1-4e1a-aea4-1104e29d2c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:37.964724 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d4236080-10a1-4e1a-aea4-1104e29d2c18 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:37.965965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d4236080-10a1-4e1a-aea4-1104e29d2c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:38.039355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 265/559] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:38.040389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4236080-10a1-4e1a-aea4-1104e29d2c18 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:38.041464 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b28efa-340a-4a99-8bff-af1729b3aa9d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:38.078878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91545b21-c118-469a-8363-85075b681220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:38.080144 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91545b21-c118-469a-8363-85075b681220 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.080706 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91545b21-c118-469a-8363-85075b681220 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.081260 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91545b21-c118-469a-8363-85075b681220 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.093567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91545b21-c118-469a-8363-85075b681220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:38.095403 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91545b21-c118-469a-8363-85075b681220 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:38.115416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-91545b21-c118-469a-8363-85075b681220 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:38.115416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:38.115416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:38.115416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:38.115416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:38.115416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:38.115416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:38.115416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:38.115416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:38.115416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:38.115416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:38.115416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:38.115416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:38.115416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:38.115416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:38.116354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:38.116354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:38.116354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:38.116354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:38.116354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:38.116354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:38.116354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:38.116354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:38.116354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:38.116354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:38.116354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:38.116354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:38.116354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:38.116354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:38.116354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:38.116354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:38.116354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:38.117228 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:38.117228 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:38.117228 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:38.117228 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:38.118237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:38.120847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 295/560] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:38.121830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91545b21-c118-469a-8363-85075b681220 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:38.144422 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:38.174731 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4236080-10a1-4e1a-aea4-1104e29d2c18 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:38.212552 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339da0b-5621-43ab-a2ca-843aa4d803ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:38.213132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339da0b-5621-43ab-a2ca-843aa4d803ff None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.213414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339da0b-5621-43ab-a2ca-843aa4d803ff None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.213687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339da0b-5621-43ab-a2ca-843aa4d803ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.237045 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91545b21-c118-469a-8363-85075b681220 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:38.272229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd28a36-0cc7-48b4-9fc9-5de6bc2f5da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:38.273396 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd28a36-0cc7-48b4-9fc9-5de6bc2f5da4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.273880 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd28a36-0cc7-48b4-9fc9-5de6bc2f5da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.274467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd28a36-0cc7-48b4-9fc9-5de6bc2f5da4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.286473 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd28a36-0cc7-48b4-9fc9-5de6bc2f5da4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:38.288033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339da0b-5621-43ab-a2ca-843aa4d803ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:38.290524 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339da0b-5621-43ab-a2ca-843aa4d803ff None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:38.296696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-ebd28a36-0cc7-48b4-9fc9-5de6bc2f5da4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:38.296696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:38.296696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:38.296696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:38.296696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:38.296696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:38.296696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:38.296696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:38.296696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:38.296696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:38.296696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:38.296696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:38.296696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:38.296696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:38.296696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:38.296696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:38.297703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:38.297703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:38.297703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:38.297703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:38.297703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:38.297703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:38.297703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:38.297703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:38.297703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:38.297703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:38.297703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:38.297703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:38.297703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:38.297703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:38.297703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:38.297703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:38.298663 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:38.298663 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:38.298663 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:38.298663 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:38.298663 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:38.298663 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:38.298663 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:38.298663 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:38.298663 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:38.298663 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:38.298663 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:38.298663 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:38.301444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 296/561] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:38.302632 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd28a36-0cc7-48b4-9fc9-5de6bc2f5da4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:38.361591 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:38.391681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:38.413090 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 266/562] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:38.413090 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339da0b-5621-43ab-a2ca-843aa4d803ff None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:38.421897 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd28a36-0cc7-48b4-9fc9-5de6bc2f5da4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:38.457239 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd38961-2a49-4137-b756-4858f3ed4b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:38.457903 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd38961-2a49-4137-b756-4858f3ed4b84 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.458231 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd38961-2a49-4137-b756-4858f3ed4b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.458508 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd38961-2a49-4137-b756-4858f3ed4b84 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.459405 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd38961-2a49-4137-b756-4858f3ed4b84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:38.459858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd38961-2a49-4137-b756-4858f3ed4b84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:38.461343 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd38961-2a49-4137-b756-4858f3ed4b84 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:38.474625 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 297/563] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:38.478244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a06d98-7351-4783-8523-9429b38b1eff None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.478500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a06d98-7351-4783-8523-9429b38b1eff None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.478765 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a06d98-7351-4783-8523-9429b38b1eff None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.480047 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 298/564] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:38.480736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a06d98-7351-4783-8523-9429b38b1eff None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:38.518855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339da0b-5621-43ab-a2ca-843aa4d803ff None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:38.557565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1d6dc-de21-4f1d-9a13-57e82a7a5584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:38.558241 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1d6dc-de21-4f1d-9a13-57e82a7a5584 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.558518 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1d6dc-de21-4f1d-9a13-57e82a7a5584 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.558788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1d6dc-de21-4f1d-9a13-57e82a7a5584 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.559529 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1d6dc-de21-4f1d-9a13-57e82a7a5584 None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:38.559881 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1d6dc-de21-4f1d-9a13-57e82a7a5584 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:38.561267 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1d6dc-de21-4f1d-9a13-57e82a7a5584 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:38.569125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 267/565] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:38.569884 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1d6dc-de21-4f1d-9a13-57e82a7a5584 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:38.584149 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a06d98-7351-4783-8523-9429b38b1eff None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:38.592653 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:38.617758 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d44c4d-b05e-4134-9686-3fe694811655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:38.618558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d44c4d-b05e-4134-9686-3fe694811655 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.618836 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d44c4d-b05e-4134-9686-3fe694811655 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.619087 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d44c4d-b05e-4134-9686-3fe694811655 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.622104 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:38.630488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d44c4d-b05e-4134-9686-3fe694811655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:38.631993 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d44c4d-b05e-4134-9686-3fe694811655 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:38.649356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-88d44c4d-b05e-4134-9686-3fe694811655 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:38.649356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:38.649356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:38.649356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:38.649356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:38.649356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:38.649356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:38.649356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:38.649356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:38.649356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:38.649356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:38.649356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:38.649356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:38.649356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:38.649356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:38.650359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:38.650359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:38.650359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:38.650359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:38.650359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:38.650359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:38.650359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:38.650359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:38.650359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:38.650359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:38.650359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:38.650359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:38.650359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:38.650359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:38.650359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:38.650359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:38.650359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:38.651251 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:38.651251 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:38.651251 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:38.651251 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:38.651251 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 299/566] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:38.652171 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdda72-1cfd-437c-a8a5-36b974004168 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.652451 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdda72-1cfd-437c-a8a5-36b974004168 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.652719 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdda72-1cfd-437c-a8a5-36b974004168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.669074 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-51bdda72-1cfd-437c-a8a5-36b974004168 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:38.670249 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-51bdda72-1cfd-437c-a8a5-36b974004168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:38.677351 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1d6dc-de21-4f1d-9a13-57e82a7a5584 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:38.711885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e877a44-15c2-45a9-ad53-ff0dfd3abeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:38.712691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e877a44-15c2-45a9-ad53-ff0dfd3abeec None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.712973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e877a44-15c2-45a9-ad53-ff0dfd3abeec None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.713246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e877a44-15c2-45a9-ad53-ff0dfd3abeec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.720120 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e877a44-15c2-45a9-ad53-ff0dfd3abeec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:38.721677 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e877a44-15c2-45a9-ad53-ff0dfd3abeec None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:38.736070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-1e877a44-15c2-45a9-ad53-ff0dfd3abeec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:38.736070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:38.736070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:38.736070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:38.736070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:38.736070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:38.736070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:38.736070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:38.736070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:38.736070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:38.736070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:38.736070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:38.736070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:38.736070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:38.736070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:38.737146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:38.737146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:38.737146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:38.737146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:38.737146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:38.737146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:38.737146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:38.737146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:38.737146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:38.737146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:38.737146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:38.737146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:38.737146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:38.737146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:38.737146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:38.737146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:38.737146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:38.738042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:38.738042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:38.738042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:38.738042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:38.738042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:38.738042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:38.738042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:38.738042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:38.738042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 268/567] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:38.738675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510dd8e-556e-4811-a994-96604e356797 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.738948 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510dd8e-556e-4811-a994-96604e356797 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.739224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510dd8e-556e-4811-a994-96604e356797 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.740491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 269/568] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:38.741162 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510dd8e-556e-4811-a994-96604e356797 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:38.743459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 300/569] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:38.744608 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bdda72-1cfd-437c-a8a5-36b974004168 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:38.828696 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:38.845291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bdda72-1cfd-437c-a8a5-36b974004168 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:38.848767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510dd8e-556e-4811-a994-96604e356797 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:38.862240 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:38.878738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f0e45-0915-4ca8-80a0-633d51b29fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:38.879409 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f0e45-0915-4ca8-80a0-633d51b29fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.879720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f0e45-0915-4ca8-80a0-633d51b29fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.879994 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f0e45-0915-4ca8-80a0-633d51b29fb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.880707 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f0e45-0915-4ca8-80a0-633d51b29fb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:38.881010 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f0e45-0915-4ca8-80a0-633d51b29fb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:38.882184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e29ab3-fd2b-4525-a913-7269a7f81df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:38.882399 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f0e45-0915-4ca8-80a0-633d51b29fb3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:38.882980 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e29ab3-fd2b-4525-a913-7269a7f81df9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.883262 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e29ab3-fd2b-4525-a913-7269a7f81df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.883539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e29ab3-fd2b-4525-a913-7269a7f81df9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.893938 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e29ab3-fd2b-4525-a913-7269a7f81df9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:38.894049 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 301/570] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:38.894816 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f0e45-0915-4ca8-80a0-633d51b29fb3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:38.895381 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e29ab3-fd2b-4525-a913-7269a7f81df9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:38.913800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-84e29ab3-fd2b-4525-a913-7269a7f81df9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:38.913800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:38.913800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:38.913800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:38.913800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:38.913800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:38.913800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:38.913800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:38.913800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:38.913800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:38.913800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:38.913800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:38.913800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:38.913800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:38.913800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:38.914747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:38.914747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:38.914747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:38.914747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:38.914747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:38.914747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:38.914747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:38.914747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:38.914747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:38.914747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:38.914747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:38.914747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:38.914747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:38.914747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:38.914747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:38.914747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:38.914747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:38.915703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:38.915703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:38.915703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:38.915703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:38.915703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 270/571] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:38.916533 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2d80b-1e44-4bfa-a478-32f912f3c31e None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.916817 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2d80b-1e44-4bfa-a478-32f912f3c31e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.917091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2d80b-1e44-4bfa-a478-32f912f3c31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.933912 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-4fc2d80b-1e44-4bfa-a478-32f912f3c31e None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:38.935129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4fc2d80b-1e44-4bfa-a478-32f912f3c31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:39.001086 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f0e45-0915-4ca8-80a0-633d51b29fb3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:39.003361 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 271/572] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:39.004236 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2d80b-1e44-4bfa-a478-32f912f3c31e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:39.039730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459d04f-2c52-42c7-922f-493203fdf446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:39.040433 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459d04f-2c52-42c7-922f-493203fdf446 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:39.040718 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459d04f-2c52-42c7-922f-493203fdf446 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:39.040991 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459d04f-2c52-42c7-922f-493203fdf446 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:39.041772 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459d04f-2c52-42c7-922f-493203fdf446 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:39.042085 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459d04f-2c52-42c7-922f-493203fdf446 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:39.043529 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459d04f-2c52-42c7-922f-493203fdf446 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:39.055311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 302/573] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:39.056118 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459d04f-2c52-42c7-922f-493203fdf446 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:39.064842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:39.096657 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:39.103465 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2d80b-1e44-4bfa-a478-32f912f3c31e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:39.138570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a82b1a-3419-4fd0-b6bb-79568a8a76c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:39.139373 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a82b1a-3419-4fd0-b6bb-79568a8a76c3 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:39.139724 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a82b1a-3419-4fd0-b6bb-79568a8a76c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:39.139952 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a82b1a-3419-4fd0-b6bb-79568a8a76c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:39.150503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a82b1a-3419-4fd0-b6bb-79568a8a76c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:39.154833 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459d04f-2c52-42c7-922f-493203fdf446 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:39.158558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-37a82b1a-3419-4fd0-b6bb-79568a8a76c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:39.158558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:39.158558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:39.158558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:39.158558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:39.158558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:39.158558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:39.158558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:39.158558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:39.158558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:39.158558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:39.158558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:39.158558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:39.158558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:39.158558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:39.158558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:39.160451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:39.160451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:39.160451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:39.160451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:39.160451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:39.160451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:39.160451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:39.160451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:39.160451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:39.160451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:39.160451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:39.160451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:39.160451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:39.160451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:39.160451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:39.160451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:39.162054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:39.162054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:39.162054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:39.162054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:39.162054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:39.162054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:39.162054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:39.162054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:39.162054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:39.162054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:39.162054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:39.162054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:39.162054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 272/574] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:39.162054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a82b1a-3419-4fd0-b6bb-79568a8a76c3 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:39.193653 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb062b3-c043-47ef-87f5-609e5a7b324c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:39.194717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb062b3-c043-47ef-87f5-609e5a7b324c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:39.195001 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb062b3-c043-47ef-87f5-609e5a7b324c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:39.195273 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb062b3-c043-47ef-87f5-609e5a7b324c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:39.202139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb062b3-c043-47ef-87f5-609e5a7b324c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:39.203594 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb062b3-c043-47ef-87f5-609e5a7b324c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:39.219949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-beb062b3-c043-47ef-87f5-609e5a7b324c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:39.219949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:39.219949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:39.219949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:39.219949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:39.219949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:39.219949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:39.219949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:39.219949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:39.219949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:39.219949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:39.219949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:39.219949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:39.219949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:39.219949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:39.220977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:39.220977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:39.220977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:39.220977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:39.220977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:39.220977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:39.220977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:39.220977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:39.220977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:39.220977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:39.220977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:39.220977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:39.220977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:39.220977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:39.220977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:39.220977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:39.220977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:39.221968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:39.221968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:39.221968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:39.221968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:39.221968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:39.221968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:39.221968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:39.221968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:39.221968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 303/575] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:39.222708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb062b3-c043-47ef-87f5-609e5a7b324c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:39.269489 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a82b1a-3419-4fd0-b6bb-79568a8a76c3 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:39.291879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:39.303721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1591b6d-f19f-4775-91ac-dde14bba02b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:39.304419 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1591b6d-f19f-4775-91ac-dde14bba02b5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:39.304687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1591b6d-f19f-4775-91ac-dde14bba02b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:39.304955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1591b6d-f19f-4775-91ac-dde14bba02b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:39.305674 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1591b6d-f19f-4775-91ac-dde14bba02b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:39.305968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1591b6d-f19f-4775-91ac-dde14bba02b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:39.307360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1591b6d-f19f-4775-91ac-dde14bba02b5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:39.318941 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 273/576] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:39.319722 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1591b6d-f19f-4775-91ac-dde14bba02b5 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:39.325935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:39.330526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb062b3-c043-47ef-87f5-609e5a7b324c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:39.365837 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca9ee7-7371-4a2a-90dc-ed8c9b791d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:39.366641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca9ee7-7371-4a2a-90dc-ed8c9b791d89 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:39.366921 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca9ee7-7371-4a2a-90dc-ed8c9b791d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:39.367192 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca9ee7-7371-4a2a-90dc-ed8c9b791d89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:39.378188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca9ee7-7371-4a2a-90dc-ed8c9b791d89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:39.385952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-7bca9ee7-7371-4a2a-90dc-ed8c9b791d89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:39.385952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:39.385952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:39.385952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:39.385952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:39.385952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:39.385952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:39.385952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:39.385952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:39.385952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:39.385952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:39.385952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:39.385952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:39.385952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:39.385952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:39.385952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:39.386900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:39.386900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:39.386900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:39.386900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:39.386900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:39.386900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:39.386900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:39.386900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:39.386900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:39.386900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:39.386900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:39.386900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:39.386900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:39.386900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:39.386900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:39.386900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:39.387773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:39.387773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:39.387773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:39.387773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:39.387773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:39.387773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:39.387773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:39.387773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:39.387773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:39.387773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:39.387773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:39.387773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:39.387773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 304/577] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:39.388454 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca9ee7-7371-4a2a-90dc-ed8c9b791d89 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:39.423870 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1591b6d-f19f-4775-91ac-dde14bba02b5 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:39.457460 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a98a371-8dd3-41cb-a9bf-c6433b655c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:39.458358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98a371-8dd3-41cb-a9bf-c6433b655c43 None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:39.458642 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98a371-8dd3-41cb-a9bf-c6433b655c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:39.458916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98a371-8dd3-41cb-a9bf-c6433b655c43 None admin] PATH_INFO: `/v3/projects/dfc29b021d1646819137c89e93f33601/users/8d7fe01ef64f46f6a0e77d5945219b33/roles/fb120deba4284112b12de1a251b7da76` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:39.473280 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a98a371-8dd3-41cb-a9bf-c6433b655c43 None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc29b021d1646819137c89e93f33601, user_id=8d7fe01ef64f46f6a0e77d5945219b33, role_id=fb120deba4284112b12de1a251b7da76)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:39.474730 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a98a371-8dd3-41cb-a9bf-c6433b655c43 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:39.488857 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 274/578] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:03:39 2022] PUT /identity/v3/projects/dfc29b021d1646819137c89e93f33601/users/8d7fe01ef64f46f6a0e77d5945219b33/roles/fb120deba4284112b12de1a251b7da76 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:39.489665 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a98a371-8dd3-41cb-a9bf-c6433b655c43 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:39.489770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca9ee7-7371-4a2a-90dc-ed8c9b791d89 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:39.536297 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad506ece-71b6-4dc8-ae00-a7ce2f65f28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:39.537014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad506ece-71b6-4dc8-ae00-a7ce2f65f28a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:39.537299 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad506ece-71b6-4dc8-ae00-a7ce2f65f28a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:39.537554 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad506ece-71b6-4dc8-ae00-a7ce2f65f28a None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:39.538438 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad506ece-71b6-4dc8-ae00-a7ce2f65f28a None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:39.539024 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:39.539447 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad506ece-71b6-4dc8-ae00-a7ce2f65f28a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:39.540954 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad506ece-71b6-4dc8-ae00-a7ce2f65f28a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:39.555486 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 305/579] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:39.555486 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad506ece-71b6-4dc8-ae00-a7ce2f65f28a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:39.580595 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:39.626525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a98a371-8dd3-41cb-a9bf-c6433b655c43 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:39.668311 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc60444-087b-49ee-a782-5666b35310c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:39.669721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc60444-087b-49ee-a782-5666b35310c6 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:39.670461 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc60444-087b-49ee-a782-5666b35310c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:39.670916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc60444-087b-49ee-a782-5666b35310c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:39.671983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc60444-087b-49ee-a782-5666b35310c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:39.672157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad506ece-71b6-4dc8-ae00-a7ce2f65f28a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:39.672638 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc60444-087b-49ee-a782-5666b35310c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:39.675310 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc60444-087b-49ee-a782-5666b35310c6 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:39.689640 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 275/580] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:39.694471 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc60444-087b-49ee-a782-5666b35310c6 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:39.714857 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81adc0-386a-4c01-aebf-3d763af191f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:39.715709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81adc0-386a-4c01-aebf-3d763af191f3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:39.715998 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81adc0-386a-4c01-aebf-3d763af191f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:39.716272 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81adc0-386a-4c01-aebf-3d763af191f3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:39.724387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81adc0-386a-4c01-aebf-3d763af191f3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:39.725862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81adc0-386a-4c01-aebf-3d763af191f3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:39.740548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-9c81adc0-386a-4c01-aebf-3d763af191f3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 27 21:03:39.740548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:39.740548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:39.740548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:39.740548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:39.740548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:39.740548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:39.740548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:39.740548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:39.740548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:39.740548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:39.740548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:39.740548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:39.740548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:39.740548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:39.740548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:39.741565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:39.741565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:39.741565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:39.741565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:39.741565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:39.741565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:39.741565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:39.741565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:39.741565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:39.741565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:39.741565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:39.741565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:39.741565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:39.741565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:39.741565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:39.741565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:39.741565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:39.741565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:39.742497 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:39.742497 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:39.742497 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:39.742497 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:39.742497 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:39.742497 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:39.742497 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:39.742497 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:39.742497 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 27 21:03:39.742497 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:39.742497 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 306/581] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:39.743072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81adc0-386a-4c01-aebf-3d763af191f3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:39.788412 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:39.798007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc60444-087b-49ee-a782-5666b35310c6 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:39.831485 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:39.832525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570ed67-d07f-4012-bd59-de6394882842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:39.833350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570ed67-d07f-4012-bd59-de6394882842 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:39.833532 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570ed67-d07f-4012-bd59-de6394882842 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:39.833869 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570ed67-d07f-4012-bd59-de6394882842 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:39.834603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570ed67-d07f-4012-bd59-de6394882842 None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:39.834938 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570ed67-d07f-4012-bd59-de6394882842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:39.836375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570ed67-d07f-4012-bd59-de6394882842 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:39.849028 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 276/582] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:39.849028 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570ed67-d07f-4012-bd59-de6394882842 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:39.857664 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81adc0-386a-4c01-aebf-3d763af191f3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:39.894977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176170d-40c3-4e4a-bd1a-63b5aa85a32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:39.895820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176170d-40c3-4e4a-bd1a-63b5aa85a32f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:39.896101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176170d-40c3-4e4a-bd1a-63b5aa85a32f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:39.896409 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176170d-40c3-4e4a-bd1a-63b5aa85a32f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:39.907192 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176170d-40c3-4e4a-bd1a-63b5aa85a32f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:39.909041 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176170d-40c3-4e4a-bd1a-63b5aa85a32f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:39.928969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f176170d-40c3-4e4a-bd1a-63b5aa85a32f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:39.928969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:39.928969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:39.928969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:39.928969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:39.928969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:39.928969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:39.928969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:39.928969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:39.928969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:39.928969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:39.928969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:39.928969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:39.928969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:39.928969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:39.929935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:39.929935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:39.929935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:39.929935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:39.929935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:39.929935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:39.929935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:39.929935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:39.929935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:39.929935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:39.929935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:39.929935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:39.929935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:39.929935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:39.929935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:39.929935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:39.929935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:39.930802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:39.930802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:39.930802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:39.930802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:39.930802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 307/583] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:39.931590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176170d-40c3-4e4a-bd1a-63b5aa85a32f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:39.956539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570ed67-d07f-4012-bd59-de6394882842 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:39.996998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2f09b-86f1-464d-b3ce-13973c9bd5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:39.997837 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2f09b-86f1-464d-b3ce-13973c9bd5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:39.998124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2f09b-86f1-464d-b3ce-13973c9bd5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:39.998564 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2f09b-86f1-464d-b3ce-13973c9bd5a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:39.999730 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2f09b-86f1-464d-b3ce-13973c9bd5a9 None admin] RBAC: Adding query filter params (role.id=fb120deba4284112b12de1a251b7da76, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=f28acb0322114aebafcf62bdf9a346df) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:40.000189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2f09b-86f1-464d-b3ce-13973c9bd5a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:40.002006 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2f09b-86f1-464d-b3ce-13973c9bd5a9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:40.019656 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 277/584] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:39 2022] GET /identity/v3/role_assignments?user.id=f28acb0322114aebafcf62bdf9a346df&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=fb120deba4284112b12de1a251b7da76 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:40.019656 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2f09b-86f1-464d-b3ce-13973c9bd5a9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:40.040491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:40.064953 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176170d-40c3-4e4a-bd1a-63b5aa85a32f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:40.100817 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:40.107057 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38fea8-5933-43ee-acae-d2a8bbb00d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:40.108049 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38fea8-5933-43ee-acae-d2a8bbb00d3c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:40.108523 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38fea8-5933-43ee-acae-d2a8bbb00d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:40.108962 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38fea8-5933-43ee-acae-d2a8bbb00d3c None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:40.110289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38fea8-5933-43ee-acae-d2a8bbb00d3c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:40.111185 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38fea8-5933-43ee-acae-d2a8bbb00d3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:40.112908 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38fea8-5933-43ee-acae-d2a8bbb00d3c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:40.133521 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 308/585] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Jun 27 21:03:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:40.134779 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38fea8-5933-43ee-acae-d2a8bbb00d3c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:40.152491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2f09b-86f1-464d-b3ce-13973c9bd5a9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:40.194045 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9857e-de52-4297-a6c0-0c5eba56c030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:40.194891 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9857e-de52-4297-a6c0-0c5eba56c030 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:40.195176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9857e-de52-4297-a6c0-0c5eba56c030 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:40.195431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9857e-de52-4297-a6c0-0c5eba56c030 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:40.206548 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f9857e-de52-4297-a6c0-0c5eba56c030 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:40.208200 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f9857e-de52-4297-a6c0-0c5eba56c030 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:40.225994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-90f9857e-de52-4297-a6c0-0c5eba56c030 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:40.225994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:40.225994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:40.225994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:40.225994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:40.225994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:40.225994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:40.225994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:40.225994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:40.225994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:40.225994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:40.225994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:40.225994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:40.225994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:40.225994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:40.226933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:40.226933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:40.226933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:40.226933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:40.226933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:40.226933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:40.226933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:40.226933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:40.226933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:40.226933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:40.226933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:40.226933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:40.226933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:40.226933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:40.226933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:40.226933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:40.226933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:40.228150 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:40.228150 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:40.228150 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:40.228150 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:40.228150 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 278/586] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:40.228434 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9857e-de52-4297-a6c0-0c5eba56c030 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:40.256694 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38fea8-5933-43ee-acae-d2a8bbb00d3c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:40.296749 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7e1f7-5eee-4fac-9091-ba8a57099d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:40.298015 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7e1f7-5eee-4fac-9091-ba8a57099d08 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:40.298817 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7e1f7-5eee-4fac-9091-ba8a57099d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:40.299309 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7e1f7-5eee-4fac-9091-ba8a57099d08 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:40.300799 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7e1f7-5eee-4fac-9091-ba8a57099d08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:40.301317 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7e1f7-5eee-4fac-9091-ba8a57099d08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:40.303411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7e1f7-5eee-4fac-9091-ba8a57099d08 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:40.310661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:40.322475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 309/587] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:40.322475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7e1f7-5eee-4fac-9091-ba8a57099d08 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:40.351048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9857e-de52-4297-a6c0-0c5eba56c030 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:40.383694 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:40.389446 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e41bd-7b82-4ac8-9cc0-07c3d9002e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:40.390744 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e41bd-7b82-4ac8-9cc0-07c3d9002e90 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:40.391701 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e41bd-7b82-4ac8-9cc0-07c3d9002e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:40.392093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e41bd-7b82-4ac8-9cc0-07c3d9002e90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:40.404538 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e41bd-7b82-4ac8-9cc0-07c3d9002e90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:40.406299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e41bd-7b82-4ac8-9cc0-07c3d9002e90 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:40.425823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-218e41bd-7b82-4ac8-9cc0-07c3d9002e90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:40.425823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:40.425823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:40.425823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:40.425823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:40.425823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:40.425823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:40.425823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:40.425823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:40.425823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:40.425823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:40.425823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:40.425823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:40.425823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:40.425823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:40.427389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:40.427389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:40.427389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:40.427389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:40.427389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:40.427389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:40.427389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:40.427389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:40.427389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:40.427389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:40.427389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:40.427389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:40.427389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:40.427389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:40.427389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:40.427389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:40.427389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:40.428476 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:40.428476 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:40.428476 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:40.428476 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:40.429343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 279/588] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:40.430193 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e41bd-7b82-4ac8-9cc0-07c3d9002e90 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:40.447904 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7e1f7-5eee-4fac-9091-ba8a57099d08 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:40.484663 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737736a-0d9f-4e59-b320-95969fcedbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:40.485447 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737736a-0d9f-4e59-b320-95969fcedbbc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:40.485838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737736a-0d9f-4e59-b320-95969fcedbbc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:40.486477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737736a-0d9f-4e59-b320-95969fcedbbc None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:40.487131 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f737736a-0d9f-4e59-b320-95969fcedbbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:40.487590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f737736a-0d9f-4e59-b320-95969fcedbbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:40.489235 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f737736a-0d9f-4e59-b320-95969fcedbbc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:40.510492 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 310/589] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:40.511371 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737736a-0d9f-4e59-b320-95969fcedbbc None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:40.553530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e41bd-7b82-4ac8-9cc0-07c3d9002e90 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:40.579747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:40.596445 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d481ff5-d851-4cda-9aa1-9e545ba71834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:40.598131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d481ff5-d851-4cda-9aa1-9e545ba71834 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:40.598979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d481ff5-d851-4cda-9aa1-9e545ba71834 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:40.599458 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d481ff5-d851-4cda-9aa1-9e545ba71834 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:40.610440 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d481ff5-d851-4cda-9aa1-9e545ba71834 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:40.612589 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d481ff5-d851-4cda-9aa1-9e545ba71834 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:40.630940 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737736a-0d9f-4e59-b320-95969fcedbbc None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:40.631822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-3d481ff5-d851-4cda-9aa1-9e545ba71834 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 21:03:40.631822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:40.631822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:40.631822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:40.631822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:40.631822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:40.631822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:40.631822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:40.631822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:40.631822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:40.631822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:40.631822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:40.631822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:40.631822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:40.631822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:40.631822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:40.632741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:40.632741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:40.632741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:40.632741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:40.632741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:40.632741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:40.632741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:40.632741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:40.632741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:40.632741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:40.632741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:40.632741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:40.632741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:40.632741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:40.632741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:40.632741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:40.632741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:40.632741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:40.633683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:40.633683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:40.633683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:40.633683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:40.633683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:40.633683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:40.633683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:40.633683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:40.633683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 21:03:40.633683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:40.641615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 280/590] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:40.642694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d481ff5-d851-4cda-9aa1-9e545ba71834 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:40.646962 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:40.672544 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b9fbe-3c43-46b5-957c-99bf68d15173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:40.673352 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b9fbe-3c43-46b5-957c-99bf68d15173 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:40.673753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b9fbe-3c43-46b5-957c-99bf68d15173 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:40.674118 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b9fbe-3c43-46b5-957c-99bf68d15173 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:40.675025 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b9fbe-3c43-46b5-957c-99bf68d15173 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:40.675444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b9fbe-3c43-46b5-957c-99bf68d15173 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:40.677063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b9fbe-3c43-46b5-957c-99bf68d15173 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:40.689892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 311/591] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:40.691048 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b9fbe-3c43-46b5-957c-99bf68d15173 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:40.750933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d481ff5-d851-4cda-9aa1-9e545ba71834 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:40.786913 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b2421-ce08-42b2-af1c-b85c50abcd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:40.788032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b2421-ce08-42b2-af1c-b85c50abcd5d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:40.788463 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b2421-ce08-42b2-af1c-b85c50abcd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:40.788887 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b2421-ce08-42b2-af1c-b85c50abcd5d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:40.797753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b9fbe-3c43-46b5-957c-99bf68d15173 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:40.798911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947b2421-ce08-42b2-af1c-b85c50abcd5d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:40.801567 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947b2421-ce08-42b2-af1c-b85c50abcd5d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:40.819931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-947b2421-ce08-42b2-af1c-b85c50abcd5d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 21:03:40.819931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:40.819931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:40.819931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:40.819931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:40.819931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:40.819931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:40.819931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:40.819931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:40.819931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:40.819931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:40.819931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:40.819931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:40.819931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:40.819931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:40.819931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:40.820856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:40.820856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:40.820856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:40.820856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:40.820856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:40.820856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:40.820856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:40.820856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:40.820856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:40.820856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:40.820856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:40.820856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:40.820856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:40.820856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:40.820856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:40.820856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:40.820856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:40.820856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:40.821802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:40.821802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:40.821802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:40.821802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:40.821802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:40.821802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:40.821802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:40.821802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:40.821802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 21:03:40.821802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:40.824769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 281/592] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:40.825925 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b2421-ce08-42b2-af1c-b85c50abcd5d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:40.835504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e94235-cf65-4035-9ce2-b7793ed46e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:40.836322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e94235-cf65-4035-9ce2-b7793ed46e48 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:40.836692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e94235-cf65-4035-9ce2-b7793ed46e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:40.837271 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e94235-cf65-4035-9ce2-b7793ed46e48 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:40.838101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e94235-cf65-4035-9ce2-b7793ed46e48 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:40.838502 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e94235-cf65-4035-9ce2-b7793ed46e48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:40.840212 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e94235-cf65-4035-9ce2-b7793ed46e48 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:40.862932 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 312/593] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Jun 27 21:03:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:40.862932 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e94235-cf65-4035-9ce2-b7793ed46e48 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:40.884322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:40.891514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:40.956922 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b2421-ce08-42b2-af1c-b85c50abcd5d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:40.969082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e94235-cf65-4035-9ce2-b7793ed46e48 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:40.998760 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc3eeb-1d8e-4d58-86b1-1b3082c2fffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:40.999925 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc3eeb-1d8e-4d58-86b1-1b3082c2fffe None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.000213 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc3eeb-1d8e-4d58-86b1-1b3082c2fffe None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.000513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc3eeb-1d8e-4d58-86b1-1b3082c2fffe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.009565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc3eeb-1d8e-4d58-86b1-1b3082c2fffe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:41.011036 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc3eeb-1d8e-4d58-86b1-1b3082c2fffe None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:41.013735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abbcdf-315b-4cad-a201-c4b8ceab5fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:41.014413 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abbcdf-315b-4cad-a201-c4b8ceab5fda None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.014689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abbcdf-315b-4cad-a201-c4b8ceab5fda None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.014963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abbcdf-315b-4cad-a201-c4b8ceab5fda None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.015822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abbcdf-315b-4cad-a201-c4b8ceab5fda None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:41.016139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abbcdf-315b-4cad-a201-c4b8ceab5fda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:41.017683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abbcdf-315b-4cad-a201-c4b8ceab5fda None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:41.033829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-bbcc3eeb-1d8e-4d58-86b1-1b3082c2fffe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:41.033829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:41.033829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:41.033829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:41.033829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:41.033829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:41.033829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:41.033829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:41.033829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:41.033829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:41.033829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:41.033829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:41.033829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:41.033829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:41.033829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:41.034763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:41.034763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:41.034763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:41.034763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:41.034763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:41.034763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:41.034763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:41.034763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:41.034763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:41.034763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:41.034763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:41.034763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:41.034763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:41.034763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:41.034763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:41.034763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:41.034763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:41.036278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:41.036278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:41.036278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:41.036278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:41.036278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:41.036278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:41.036278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:41.036278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:41.036278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 282/594] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:41.036278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc3eeb-1d8e-4d58-86b1-1b3082c2fffe None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:41.043903 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 313/595] 10.222.0.27 () {56 vars in 1139 bytes} [Mon Jun 27 21:03:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:41.045159 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abbcdf-315b-4cad-a201-c4b8ceab5fda None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:41.143777 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc3eeb-1d8e-4d58-86b1-1b3082c2fffe None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:41.145512 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:41.162330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:41.185170 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abbcdf-315b-4cad-a201-c4b8ceab5fda None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:41.188010 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5deb2be-7eeb-43ad-b354-aa45d97f65c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:41.189298 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5deb2be-7eeb-43ad-b354-aa45d97f65c7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.189800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5deb2be-7eeb-43ad-b354-aa45d97f65c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.190275 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5deb2be-7eeb-43ad-b354-aa45d97f65c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.201776 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5deb2be-7eeb-43ad-b354-aa45d97f65c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:41.203638 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5deb2be-7eeb-43ad-b354-aa45d97f65c7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:41.223211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-e5deb2be-7eeb-43ad-b354-aa45d97f65c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:41.223211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:41.223211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:41.223211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:41.223211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:41.223211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:41.223211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:41.223211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:41.223211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:41.223211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:41.223211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:41.223211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:41.223211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:41.223211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:41.223211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:41.224352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:41.224352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:41.224352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:41.224352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:41.224352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:41.224352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:41.224352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:41.224352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:41.224352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:41.224352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:41.224352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:41.224352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:41.224352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:41.224352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:41.224352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:41.224352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:41.224352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:41.225211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:41.225211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:41.225211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:41.225211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:41.225421 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267b8da-ea8d-4fa5-bb2a-d58132a83da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:41.225861 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267b8da-ea8d-4fa5-bb2a-d58132a83da6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.226015 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267b8da-ea8d-4fa5-bb2a-d58132a83da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.226296 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267b8da-ea8d-4fa5-bb2a-d58132a83da6 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.227095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267b8da-ea8d-4fa5-bb2a-d58132a83da6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:41.227406 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267b8da-ea8d-4fa5-bb2a-d58132a83da6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:41.228422 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 283/596] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:41.229393 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5deb2be-7eeb-43ad-b354-aa45d97f65c7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:41.232264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267b8da-ea8d-4fa5-bb2a-d58132a83da6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:41.245920 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 314/597] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:41.247263 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267b8da-ea8d-4fa5-bb2a-d58132a83da6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:41.338542 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5deb2be-7eeb-43ad-b354-aa45d97f65c7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:41.351222 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267b8da-ea8d-4fa5-bb2a-d58132a83da6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:41.373892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc09d9-cc7d-4923-be9b-55d3d805cfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:41.374776 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc09d9-cc7d-4923-be9b-55d3d805cfff None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.374987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc09d9-cc7d-4923-be9b-55d3d805cfff None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.375265 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc09d9-cc7d-4923-be9b-55d3d805cfff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.386943 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc09d9-cc7d-4923-be9b-55d3d805cfff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:41.387533 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af20f41-dc33-48ab-a570-f47f35a6550d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:41.388454 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc09d9-cc7d-4923-be9b-55d3d805cfff None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:41.388835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af20f41-dc33-48ab-a570-f47f35a6550d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.389307 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af20f41-dc33-48ab-a570-f47f35a6550d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.389786 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af20f41-dc33-48ab-a570-f47f35a6550d None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.390693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af20f41-dc33-48ab-a570-f47f35a6550d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:41.391186 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af20f41-dc33-48ab-a570-f47f35a6550d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:41.392812 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af20f41-dc33-48ab-a570-f47f35a6550d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:41.405743 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 315/598] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:41.406747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af20f41-dc33-48ab-a570-f47f35a6550d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:41.409306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-1fdc09d9-cc7d-4923-be9b-55d3d805cfff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:41.409306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:41.409306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:41.409306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:41.409306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:41.409306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:41.409306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:41.409306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:41.409306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:41.409306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:41.409306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:41.409306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:41.409306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:41.409306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:41.409306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:41.410131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:41.410131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:41.410131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:41.410131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:41.410131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:41.410131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:41.410131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:41.410131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:41.410131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:41.410131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:41.410131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:41.410131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:41.410131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:41.410131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:41.410131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:41.410131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:41.410131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:41.410985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:41.410985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:41.410985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:41.410985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:41.413251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 284/599] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:41.414452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc09d9-cc7d-4923-be9b-55d3d805cfff None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:41.424187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:41.428407 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:41.522987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc09d9-cc7d-4923-be9b-55d3d805cfff None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:41.523689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af20f41-dc33-48ab-a570-f47f35a6550d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:41.560576 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e0d70-4c6c-4ece-ab59-a8cbcfd82a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:41.561661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e0d70-4c6c-4ece-ab59-a8cbcfd82a78 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.562074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e0d70-4c6c-4ece-ab59-a8cbcfd82a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.562484 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e0d70-4c6c-4ece-ab59-a8cbcfd82a78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.563210 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297b5fa-8c9d-4bd9-92ed-481901808a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:41.564271 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297b5fa-8c9d-4bd9-92ed-481901808a26 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.564561 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297b5fa-8c9d-4bd9-92ed-481901808a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.564840 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297b5fa-8c9d-4bd9-92ed-481901808a26 None admin] PATH_INFO: `/v3/projects/dfc29b021d1646819137c89e93f33601/users/b43db1fd0b494a3f8588b498b585f247/roles/fb120deba4284112b12de1a251b7da76` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.571265 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e0d70-4c6c-4ece-ab59-a8cbcfd82a78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:41.573351 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e0d70-4c6c-4ece-ab59-a8cbcfd82a78 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:41.579809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6297b5fa-8c9d-4bd9-92ed-481901808a26 None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc29b021d1646819137c89e93f33601, user_id=b43db1fd0b494a3f8588b498b585f247, role_id=fb120deba4284112b12de1a251b7da76)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:41.581277 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6297b5fa-8c9d-4bd9-92ed-481901808a26 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:41.587906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-667e0d70-4c6c-4ece-ab59-a8cbcfd82a78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:41.587906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:41.587906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:41.587906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:41.587906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:41.587906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:41.587906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:41.587906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:41.587906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:41.587906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:41.587906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:41.587906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:41.587906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:41.587906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:41.587906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:41.588893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:41.588893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:41.588893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:41.588893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:41.588893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:41.588893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:41.588893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:41.588893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:41.588893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:41.588893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:41.588893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:41.588893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:41.588893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:41.588893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:41.588893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:41.588893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:41.588893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:41.589788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:41.589788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:41.589788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:41.589788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:41.589788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:41.589788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:41.589788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:41.589788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:41.593759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 285/600] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:41.594605 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e0d70-4c6c-4ece-ab59-a8cbcfd82a78 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:41.604924 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 316/601] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:03:41 2022] PUT /identity/v3/projects/dfc29b021d1646819137c89e93f33601/users/b43db1fd0b494a3f8588b498b585f247/roles/fb120deba4284112b12de1a251b7da76 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:41.604924 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75021dca-3f13-44cb-bd0b-7a5da654f843 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.604924 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75021dca-3f13-44cb-bd0b-7a5da654f843 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.604924 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75021dca-3f13-44cb-bd0b-7a5da654f843 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.604924 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 317/602] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:41.604924 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75021dca-3f13-44cb-bd0b-7a5da654f843 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:41.689160 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:41.690654 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:41.743828 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e0d70-4c6c-4ece-ab59-a8cbcfd82a78 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:41.756161 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75021dca-3f13-44cb-bd0b-7a5da654f843 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:41.785372 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a60ac5-4cb8-463b-87e0-caaa6d160f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:41.786389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a60ac5-4cb8-463b-87e0-caaa6d160f48 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.786866 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a60ac5-4cb8-463b-87e0-caaa6d160f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.787252 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a60ac5-4cb8-463b-87e0-caaa6d160f48 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.788147 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a60ac5-4cb8-463b-87e0-caaa6d160f48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:41.788671 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a60ac5-4cb8-463b-87e0-caaa6d160f48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:41.790215 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a60ac5-4cb8-463b-87e0-caaa6d160f48 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:41.795134 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd5e0c-a803-462a-ae1e-66a435748558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:41.796542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd5e0c-a803-462a-ae1e-66a435748558 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.796542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd5e0c-a803-462a-ae1e-66a435748558 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.796542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd5e0c-a803-462a-ae1e-66a435748558 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.797131 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd5e0c-a803-462a-ae1e-66a435748558 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:41.797435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd5e0c-a803-462a-ae1e-66a435748558 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:41.798946 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd5e0c-a803-462a-ae1e-66a435748558 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:41.805858 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 286/603] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:41.808135 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd5bf5-3852-4010-8c27-c91bcae4d5e9 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.809094 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd5bf5-3852-4010-8c27-c91bcae4d5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.809548 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd5bf5-3852-4010-8c27-c91bcae4d5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.811990 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 318/604] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:41.812826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd5e0c-a803-462a-ae1e-66a435748558 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:41.830799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b1dd5bf5-3852-4010-8c27-c91bcae4d5e9 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:41.832728 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b1dd5bf5-3852-4010-8c27-c91bcae4d5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:41.916388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 287/605] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:41.921697 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd5e0c-a803-462a-ae1e-66a435748558 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:41.924947 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd5bf5-3852-4010-8c27-c91bcae4d5e9 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:41.956201 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:41.964662 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd0e3c-6467-4c21-9e21-5d21cc1eef1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:41.965469 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd0e3c-6467-4c21-9e21-5d21cc1eef1c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.965754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd0e3c-6467-4c21-9e21-5d21cc1eef1c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.966038 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd0e3c-6467-4c21-9e21-5d21cc1eef1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.974188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd0e3c-6467-4c21-9e21-5d21cc1eef1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:41.976156 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd0e3c-6467-4c21-9e21-5d21cc1eef1c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:41.991496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-cdbd0e3c-6467-4c21-9e21-5d21cc1eef1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:41.991496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:41.991496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:41.991496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:41.991496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:41.991496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:41.991496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:41.991496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:41.991496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:41.991496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:41.991496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:41.991496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:41.991496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:41.991496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:41.991496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:41.992639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:41.992639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:41.992639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:41.992639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:41.992639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:41.992639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:41.992639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:41.992639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:41.992639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:41.992639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:41.992639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:41.992639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:41.992639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:41.992639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:41.992639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:41.992639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:41.992639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:41.993709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:41.993709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:41.993709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:41.993709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:41.993709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:41.993709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:41.993709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:41.993709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:41.997664 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 319/606] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:41.998389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd0e3c-6467-4c21-9e21-5d21cc1eef1c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:42.004439 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:42.059414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd5bf5-3852-4010-8c27-c91bcae4d5e9 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:42.101827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52143fe9-cb17-4537-9b75-c45ecb236b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:42.103193 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52143fe9-cb17-4537-9b75-c45ecb236b25 None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:42.103588 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52143fe9-cb17-4537-9b75-c45ecb236b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:42.103901 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52143fe9-cb17-4537-9b75-c45ecb236b25 None admin] PATH_INFO: `/v3/projects/dfc29b021d1646819137c89e93f33601/users/16fda36445d9415aae4df74f31af0601/roles/fb120deba4284112b12de1a251b7da76` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:42.108634 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd0e3c-6467-4c21-9e21-5d21cc1eef1c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:42.122260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52143fe9-cb17-4537-9b75-c45ecb236b25 None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc29b021d1646819137c89e93f33601, user_id=16fda36445d9415aae4df74f31af0601, role_id=fb120deba4284112b12de1a251b7da76)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:42.123790 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52143fe9-cb17-4537-9b75-c45ecb236b25 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:42.139455 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 288/607] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:03:41 2022] PUT /identity/v3/projects/dfc29b021d1646819137c89e93f33601/users/16fda36445d9415aae4df74f31af0601/roles/fb120deba4284112b12de1a251b7da76 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:42.145019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52143fe9-cb17-4537-9b75-c45ecb236b25 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:42.152300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052f189-a15c-4f43-91bd-fe8ea4511133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:42.153538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052f189-a15c-4f43-91bd-fe8ea4511133 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:42.154029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052f189-a15c-4f43-91bd-fe8ea4511133 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:42.154535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052f189-a15c-4f43-91bd-fe8ea4511133 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:42.169211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052f189-a15c-4f43-91bd-fe8ea4511133 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:42.178457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-3052f189-a15c-4f43-91bd-fe8ea4511133 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:42.178457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:42.178457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:42.178457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:42.178457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:42.178457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:42.178457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:42.178457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:42.178457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:42.178457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:42.178457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:42.178457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:42.178457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:42.178457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:42.178457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:42.178457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:42.179621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:42.179621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:42.179621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:42.179621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:42.179621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:42.179621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:42.179621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:42.179621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:42.179621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:42.179621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:42.179621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:42.179621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:42.179621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:42.179621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:42.179621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:42.179621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:42.180500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:42.180500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:42.180500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:42.180500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:42.180500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:42.180500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:42.180500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:42.180500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:42.180500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:42.180500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:42.180500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:42.180500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:42.183449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 320/608] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:42.185175 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f04b1-152f-45b4-9c1d-92fe257fa683 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:42.185689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f04b1-152f-45b4-9c1d-92fe257fa683 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:42.186209 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f04b1-152f-45b4-9c1d-92fe257fa683 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:42.188193 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 321/609] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:42.189432 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f04b1-152f-45b4-9c1d-92fe257fa683 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:42.233530 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:42.289140 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52143fe9-cb17-4537-9b75-c45ecb236b25 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:42.299923 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:42.315569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f04b1-152f-45b4-9c1d-92fe257fa683 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:42.337682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196add3f-e013-46e6-9298-ead055acc33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:42.338359 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196add3f-e013-46e6-9298-ead055acc33d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:42.338645 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196add3f-e013-46e6-9298-ead055acc33d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:42.338922 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196add3f-e013-46e6-9298-ead055acc33d None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:42.339772 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196add3f-e013-46e6-9298-ead055acc33d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:42.340084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196add3f-e013-46e6-9298-ead055acc33d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:42.341527 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196add3f-e013-46e6-9298-ead055acc33d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:42.354001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 289/610] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:42.355573 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71579e92-a62e-491e-96a4-e9b967ef6752 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:42.355899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71579e92-a62e-491e-96a4-e9b967ef6752 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:42.356180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71579e92-a62e-491e-96a4-e9b967ef6752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:42.368599 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc0624-6667-4026-b4bd-783a8bf580c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:42.369342 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc0624-6667-4026-b4bd-783a8bf580c6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:42.369675 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc0624-6667-4026-b4bd-783a8bf580c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:42.369963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc0624-6667-4026-b4bd-783a8bf580c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:42.370722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc0624-6667-4026-b4bd-783a8bf580c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:42.371033 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc0624-6667-4026-b4bd-783a8bf580c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:42.372457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc0624-6667-4026-b4bd-783a8bf580c6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:42.373310 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-71579e92-a62e-491e-96a4-e9b967ef6752 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:42.374519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-71579e92-a62e-491e-96a4-e9b967ef6752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:42.381440 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 322/611] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:42.382376 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc0624-6667-4026-b4bd-783a8bf580c6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:42.445752 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 290/612] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:42.446628 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71579e92-a62e-491e-96a4-e9b967ef6752 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:42.506453 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:42.509904 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc0624-6667-4026-b4bd-783a8bf580c6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:42.550197 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020c5c2-ec75-471c-b17e-1bb6e013b194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:42.551451 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020c5c2-ec75-471c-b17e-1bb6e013b194 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:42.551762 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020c5c2-ec75-471c-b17e-1bb6e013b194 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:42.552036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020c5c2-ec75-471c-b17e-1bb6e013b194 None admin] PATH_INFO: `/v3/projects/dfc29b021d1646819137c89e93f33601/users/9c7ce29db6b14176a4c9be436bcddc6e/roles/fb120deba4284112b12de1a251b7da76` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:42.569222 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020c5c2-ec75-471c-b17e-1bb6e013b194 None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc29b021d1646819137c89e93f33601, user_id=9c7ce29db6b14176a4c9be436bcddc6e, role_id=fb120deba4284112b12de1a251b7da76)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:42.570289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:42.570693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020c5c2-ec75-471c-b17e-1bb6e013b194 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:42.577943 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71579e92-a62e-491e-96a4-e9b967ef6752 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:42.590334 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 323/613] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:03:42 2022] PUT /identity/v3/projects/dfc29b021d1646819137c89e93f33601/users/9c7ce29db6b14176a4c9be436bcddc6e/roles/fb120deba4284112b12de1a251b7da76 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:42.591751 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020c5c2-ec75-471c-b17e-1bb6e013b194 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:42.649622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995dc1e-4491-472d-99b8-214efa686c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:42.650454 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995dc1e-4491-472d-99b8-214efa686c19 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:42.650739 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995dc1e-4491-472d-99b8-214efa686c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:42.651019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995dc1e-4491-472d-99b8-214efa686c19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:42.675754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995dc1e-4491-472d-99b8-214efa686c19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:42.679837 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995dc1e-4491-472d-99b8-214efa686c19 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:42.715087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-2995dc1e-4491-472d-99b8-214efa686c19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:42.715087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:42.715087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:42.715087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:42.715087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:42.715087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:42.715087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:42.715087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:42.715087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:42.715087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:42.715087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:42.715087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:42.715087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:42.715087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:42.715087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:42.716099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:42.716099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:42.716099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:42.716099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:42.716099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:42.716099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:42.716099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:42.716099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:42.716099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:42.716099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:42.716099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:42.716099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:42.716099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:42.716099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:42.716099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:42.716099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:42.716099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:42.716984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:42.716984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:42.716984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:42.716984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:42.716984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 291/614] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:42.725599 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995dc1e-4491-472d-99b8-214efa686c19 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:42.793104 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020c5c2-ec75-471c-b17e-1bb6e013b194 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:42.840186 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76de22c-1c83-4ea9-a5e5-f65b15b71fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:42.841689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76de22c-1c83-4ea9-a5e5-f65b15b71fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:42.842356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76de22c-1c83-4ea9-a5e5-f65b15b71fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:42.842776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76de22c-1c83-4ea9-a5e5-f65b15b71fa0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:42.853087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995dc1e-4491-472d-99b8-214efa686c19 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:42.856921 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76de22c-1c83-4ea9-a5e5-f65b15b71fa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:42.866678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f76de22c-1c83-4ea9-a5e5-f65b15b71fa0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:42.866678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:42.866678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:42.866678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:42.866678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:42.866678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:42.866678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:42.866678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:42.866678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:42.866678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:42.866678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:42.866678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:42.866678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:42.866678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:42.866678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:42.866678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:42.867848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:42.867848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:42.867848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:42.867848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:42.867848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:42.867848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:42.867848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:42.867848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:42.867848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:42.867848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:42.867848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:42.867848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:42.867848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:42.867848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:42.867848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:42.867848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:42.869612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:42.869612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:42.869612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:42.869612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:42.869612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:42.869612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:42.869612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:42.869612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:42.869612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:42.869612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:42.869612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:42.869612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:42.874399 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 324/615] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:42.882476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76de22c-1c83-4ea9-a5e5-f65b15b71fa0 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:42.895957 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:42.908208 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8584fa-15a4-4f79-9a81-13cf7ab664c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:42.908938 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8584fa-15a4-4f79-9a81-13cf7ab664c5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:42.909221 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8584fa-15a4-4f79-9a81-13cf7ab664c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:42.909495 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8584fa-15a4-4f79-9a81-13cf7ab664c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:42.910228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8584fa-15a4-4f79-9a81-13cf7ab664c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:42.910535 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8584fa-15a4-4f79-9a81-13cf7ab664c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:42.914776 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8584fa-15a4-4f79-9a81-13cf7ab664c5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:42.931155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 292/616] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:42.936413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8584fa-15a4-4f79-9a81-13cf7ab664c5 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:42.959986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:43.051963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76de22c-1c83-4ea9-a5e5-f65b15b71fa0 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:43.088403 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216d047-ffaf-4c45-a847-3fa4e3bb659c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:43.089370 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216d047-ffaf-4c45-a847-3fa4e3bb659c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:43.090070 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216d047-ffaf-4c45-a847-3fa4e3bb659c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:43.090355 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216d047-ffaf-4c45-a847-3fa4e3bb659c None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:43.091250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216d047-ffaf-4c45-a847-3fa4e3bb659c None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:43.091568 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216d047-ffaf-4c45-a847-3fa4e3bb659c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:43.092996 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216d047-ffaf-4c45-a847-3fa4e3bb659c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:43.102734 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 325/617] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:43.107836 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216d047-ffaf-4c45-a847-3fa4e3bb659c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:43.185598 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8584fa-15a4-4f79-9a81-13cf7ab664c5 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:43.223009 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3dc9a-f76e-47a2-9a1f-5c69ef1547f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:43.224025 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3dc9a-f76e-47a2-9a1f-5c69ef1547f9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:43.224311 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3dc9a-f76e-47a2-9a1f-5c69ef1547f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:43.224587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3dc9a-f76e-47a2-9a1f-5c69ef1547f9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:43.233740 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3dc9a-f76e-47a2-9a1f-5c69ef1547f9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:43.235567 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3dc9a-f76e-47a2-9a1f-5c69ef1547f9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:43.252031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-33f3dc9a-f76e-47a2-9a1f-5c69ef1547f9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:43.252031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:43.252031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:43.252031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:43.252031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:43.252031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:43.252031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:43.252031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:43.252031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:43.252031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:43.252031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:43.252031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:43.252031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:43.252031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:43.252031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:43.252031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:43.253001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:43.253001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:43.253001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:43.253001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:43.253001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:43.253001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:43.253001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:43.253001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:43.253001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:43.253001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:43.253001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:43.253001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:43.253001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:43.253001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:43.253001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:43.253001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:43.253001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:43.253001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:43.254005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:43.254005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:43.254005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:43.254005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:43.254005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:43.254005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:43.254005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:43.254005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:43.254005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:43.254005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:43.255755 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 293/618] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Jun 27 21:03:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:43.259657 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3dc9a-f76e-47a2-9a1f-5c69ef1547f9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:43.263948 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:43.308253 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216d047-ffaf-4c45-a847-3fa4e3bb659c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:43.375487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a6a3f-07e0-45cd-9d25-22fe95eaefe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:43.376340 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a6a3f-07e0-45cd-9d25-22fe95eaefe6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:43.376677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a6a3f-07e0-45cd-9d25-22fe95eaefe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:43.376956 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a6a3f-07e0-45cd-9d25-22fe95eaefe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:43.391377 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a6a3f-07e0-45cd-9d25-22fe95eaefe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:43.392474 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:43.393884 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a6a3f-07e0-45cd-9d25-22fe95eaefe6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:43.427281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-057a6a3f-07e0-45cd-9d25-22fe95eaefe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:43.427281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:43.427281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:43.427281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:43.427281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:43.427281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:43.427281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:43.427281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:43.427281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:43.427281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:43.427281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:43.427281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:43.427281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:43.427281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:43.427281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:43.428443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:43.428443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:43.428443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:43.428443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:43.428443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:43.428443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:43.428443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:43.428443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:43.428443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:43.428443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:43.428443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:43.428443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:43.428443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:43.428443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:43.428443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:43.428443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:43.428443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:43.429677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:43.429677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:43.429677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:43.429677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:43.432175 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 326/619] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:43.437696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a6a3f-07e0-45cd-9d25-22fe95eaefe6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:43.471577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3dc9a-f76e-47a2-9a1f-5c69ef1547f9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:43.540855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c3e97-5f0a-48e3-80ba-05d8a304c362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:43.541555 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c3e97-5f0a-48e3-80ba-05d8a304c362 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:43.541839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c3e97-5f0a-48e3-80ba-05d8a304c362 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:43.542114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c3e97-5f0a-48e3-80ba-05d8a304c362 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:43.542973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c3e97-5f0a-48e3-80ba-05d8a304c362 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:43.543287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c3e97-5f0a-48e3-80ba-05d8a304c362 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:43.544790 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c3e97-5f0a-48e3-80ba-05d8a304c362 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:43.582972 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 294/620] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Jun 27 21:03:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:43.589499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c3e97-5f0a-48e3-80ba-05d8a304c362 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:43.629811 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a6a3f-07e0-45cd-9d25-22fe95eaefe6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:43.668203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:43.682030 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead13307-1f6c-466b-8f56-c2ee495037be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:43.683084 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead13307-1f6c-466b-8f56-c2ee495037be None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:43.683571 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead13307-1f6c-466b-8f56-c2ee495037be None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:43.684099 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead13307-1f6c-466b-8f56-c2ee495037be None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:43.685076 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead13307-1f6c-466b-8f56-c2ee495037be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:43.685584 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead13307-1f6c-466b-8f56-c2ee495037be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:43.687430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead13307-1f6c-466b-8f56-c2ee495037be None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:43.702006 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 327/621] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:43.712720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead13307-1f6c-466b-8f56-c2ee495037be None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:43.741142 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c3e97-5f0a-48e3-80ba-05d8a304c362 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:43.790714 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:43.807019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196764f8-4fe0-42c1-bb31-13724ca71495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:43.807946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196764f8-4fe0-42c1-bb31-13724ca71495 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:43.808170 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196764f8-4fe0-42c1-bb31-13724ca71495 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:43.808408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196764f8-4fe0-42c1-bb31-13724ca71495 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:43.820899 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead13307-1f6c-466b-8f56-c2ee495037be None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:43.859681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f3a14-1559-47d4-9502-59a6b28bdc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:43.860824 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f3a14-1559-47d4-9502-59a6b28bdc5e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:43.861407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f3a14-1559-47d4-9502-59a6b28bdc5e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:43.861897 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f3a14-1559-47d4-9502-59a6b28bdc5e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:43.870073 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f3a14-1559-47d4-9502-59a6b28bdc5e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:43.871901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f3a14-1559-47d4-9502-59a6b28bdc5e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:43.888226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-b37f3a14-1559-47d4-9502-59a6b28bdc5e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:03:43.888226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:43.888226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:43.888226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:43.888226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:43.888226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:43.888226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:43.888226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:43.888226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:43.888226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:43.888226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:43.888226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:43.888226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:43.888226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:43.888226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:43.888226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:43.889211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:43.889211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:43.889211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:43.889211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:43.889211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:43.889211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:43.889211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:43.889211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:43.889211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:43.889211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:43.889211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:43.889211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:43.889211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:43.889211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:43.889211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:43.889211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:43.889211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:43.889211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:43.890139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:43.890139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:43.890139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:43.890139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:43.890139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:43.890139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:43.890139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:43.890139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:43.890139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:03:43.890139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:43.893077 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 328/622] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:43.902027 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f3a14-1559-47d4-9502-59a6b28bdc5e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:43.961025 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196764f8-4fe0-42c1-bb31-13724ca71495 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:43.962858 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196764f8-4fe0-42c1-bb31-13724ca71495 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:43.977496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:44.039460 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f3a14-1559-47d4-9502-59a6b28bdc5e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:44.073436 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 295/623] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:44.074889 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400d853-22d6-46bb-a87e-3e55fca738b0 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.075298 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400d853-22d6-46bb-a87e-3e55fca738b0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.075702 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400d853-22d6-46bb-a87e-3e55fca738b0 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.076136 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b00f96-e2f5-4ac9-b1e0-57cf28afa24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:44.076977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b00f96-e2f5-4ac9-b1e0-57cf28afa24e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.077400 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b00f96-e2f5-4ac9-b1e0-57cf28afa24e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.077575 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 296/624] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:44.078027 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b00f96-e2f5-4ac9-b1e0-57cf28afa24e None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.078443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400d853-22d6-46bb-a87e-3e55fca738b0 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:44.079242 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b00f96-e2f5-4ac9-b1e0-57cf28afa24e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:44.079802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b00f96-e2f5-4ac9-b1e0-57cf28afa24e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:44.081411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b00f96-e2f5-4ac9-b1e0-57cf28afa24e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:44.100549 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 329/625] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Jun 27 21:03:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:44.106315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cdefe-ef12-474e-9c7c-c7f0606f4372 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.106905 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cdefe-ef12-474e-9c7c-c7f0606f4372 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.107367 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cdefe-ef12-474e-9c7c-c7f0606f4372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.113336 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:44.127147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ea7cdefe-ef12-474e-9c7c-c7f0606f4372 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:44.129183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ea7cdefe-ef12-474e-9c7c-c7f0606f4372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:44.191295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400d853-22d6-46bb-a87e-3e55fca738b0 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:44.206480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 330/626] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:44.207641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cdefe-ef12-474e-9c7c-c7f0606f4372 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:44.239723 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:44.253639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dce0fc-ce93-4852-885b-73fc0d8e7a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:44.254566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dce0fc-ce93-4852-885b-73fc0d8e7a3d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.254923 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dce0fc-ce93-4852-885b-73fc0d8e7a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.255272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dce0fc-ce93-4852-885b-73fc0d8e7a3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.266161 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dce0fc-ce93-4852-885b-73fc0d8e7a3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:44.267801 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dce0fc-ce93-4852-885b-73fc0d8e7a3d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:44.286477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-d6dce0fc-ce93-4852-885b-73fc0d8e7a3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:44.286477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:44.286477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:44.286477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:44.286477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:44.286477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:44.286477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:44.286477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:44.286477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:44.286477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:44.286477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:44.286477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:44.286477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:44.286477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:44.286477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:44.287417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:44.287417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:44.287417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:44.287417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:44.287417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:44.287417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:44.287417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:44.287417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:44.287417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:44.287417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:44.287417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:44.287417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:44.287417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:44.287417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:44.287417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:44.287417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:44.287417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:44.288498 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:44.288498 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:44.288498 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:44.288498 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:44.292152 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 297/627] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:44.293099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dce0fc-ce93-4852-885b-73fc0d8e7a3d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:44.324029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cdefe-ef12-474e-9c7c-c7f0606f4372 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:44.362144 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4569a67-c723-4eaf-b5a8-da11a6e8f393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:44.363409 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4569a67-c723-4eaf-b5a8-da11a6e8f393 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.363897 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4569a67-c723-4eaf-b5a8-da11a6e8f393 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.364415 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4569a67-c723-4eaf-b5a8-da11a6e8f393 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.375124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:44.378015 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4569a67-c723-4eaf-b5a8-da11a6e8f393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:44.379831 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4569a67-c723-4eaf-b5a8-da11a6e8f393 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:44.400379 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dce0fc-ce93-4852-885b-73fc0d8e7a3d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:44.405754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-b4569a67-c723-4eaf-b5a8-da11a6e8f393 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:44.405754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:44.405754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:44.405754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:44.405754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:44.405754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:44.405754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:44.405754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:44.405754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:44.405754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:44.405754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:44.405754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:44.405754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:44.405754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:44.405754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:44.406669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:44.406669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:44.406669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:44.406669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:44.406669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:44.406669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:44.406669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:44.406669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:44.406669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:44.406669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:44.406669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:44.406669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:44.406669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:44.406669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:44.406669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:44.406669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:44.406669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:44.407562 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:44.407562 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:44.407562 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:44.407562 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:44.414527 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 331/628] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:44.415879 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4569a67-c723-4eaf-b5a8-da11a6e8f393 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:44.436788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a7203-9caa-41e4-a0bd-ce7b59d03433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:44.437622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a7203-9caa-41e4-a0bd-ce7b59d03433 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.438036 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a7203-9caa-41e4-a0bd-ce7b59d03433 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.438204 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a7203-9caa-41e4-a0bd-ce7b59d03433 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.449063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a7203-9caa-41e4-a0bd-ce7b59d03433 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:44.457029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-d93a7203-9caa-41e4-a0bd-ce7b59d03433 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:44.457029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:44.457029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:44.457029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:44.457029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:44.457029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:44.457029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:44.457029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:44.457029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:44.457029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:44.457029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:44.457029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:44.457029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:44.457029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:44.457029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:44.457029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:44.457949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:44.457949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:44.457949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:44.457949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:44.457949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:44.457949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:44.457949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:44.457949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:44.457949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:44.457949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:44.457949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:44.457949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:44.457949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:44.457949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:44.457949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:44.457949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:44.458814 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:44.458814 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:44.458814 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:44.458814 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:44.458814 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:44.458814 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:44.458814 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:44.458814 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:44.458814 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:44.458814 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:44.458814 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:44.458814 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:44.463317 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 298/629] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:44.464253 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a7203-9caa-41e4-a0bd-ce7b59d03433 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:44.519372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:44.545153 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4569a67-c723-4eaf-b5a8-da11a6e8f393 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:44.579783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a1777-fc84-40e9-8dea-5a94cbc19384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:44.580462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a1777-fc84-40e9-8dea-5a94cbc19384 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.580747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a1777-fc84-40e9-8dea-5a94cbc19384 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.581027 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a1777-fc84-40e9-8dea-5a94cbc19384 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.581404 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a7203-9caa-41e4-a0bd-ce7b59d03433 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:44.581849 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7a1777-fc84-40e9-8dea-5a94cbc19384 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:44.582046 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7a1777-fc84-40e9-8dea-5a94cbc19384 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:44.583444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7a1777-fc84-40e9-8dea-5a94cbc19384 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:44.595555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 332/630] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:44.596389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a1777-fc84-40e9-8dea-5a94cbc19384 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:44.622825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d7fc5-c463-4a37-86ce-83de5c98fab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:44.623860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d7fc5-c463-4a37-86ce-83de5c98fab5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.624336 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d7fc5-c463-4a37-86ce-83de5c98fab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.624863 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d7fc5-c463-4a37-86ce-83de5c98fab5 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.625905 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d7fc5-c463-4a37-86ce-83de5c98fab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:44.626400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d7fc5-c463-4a37-86ce-83de5c98fab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:44.628237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d7fc5-c463-4a37-86ce-83de5c98fab5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:44.635815 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:44.646989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 299/631] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:44.648368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43742944-1034-4dca-a43e-3cd3a3a627ef None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.648681 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43742944-1034-4dca-a43e-3cd3a3a627ef None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.648950 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43742944-1034-4dca-a43e-3cd3a3a627ef None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.651717 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 300/632] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:44.651717 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43742944-1034-4dca-a43e-3cd3a3a627ef None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:44.703961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a1777-fc84-40e9-8dea-5a94cbc19384 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:44.740906 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ac363-ebc3-49d2-aa66-e9ff74185e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:44.741683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ac363-ebc3-49d2-aa66-e9ff74185e2c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.741971 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ac363-ebc3-49d2-aa66-e9ff74185e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.742578 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ac363-ebc3-49d2-aa66-e9ff74185e2c None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.743108 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ac363-ebc3-49d2-aa66-e9ff74185e2c None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:44.743448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ac363-ebc3-49d2-aa66-e9ff74185e2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:44.744833 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ac363-ebc3-49d2-aa66-e9ff74185e2c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:44.758157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 333/633] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:44.758157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ac363-ebc3-49d2-aa66-e9ff74185e2c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:44.762563 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:44.767034 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43742944-1034-4dca-a43e-3cd3a3a627ef None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:44.804738 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6c4f6-7d15-4911-a5b6-24ee7c3d31ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:44.805607 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6c4f6-7d15-4911-a5b6-24ee7c3d31ae None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.805870 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6c4f6-7d15-4911-a5b6-24ee7c3d31ae None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.806144 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6c4f6-7d15-4911-a5b6-24ee7c3d31ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.813085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e6c4f6-7d15-4911-a5b6-24ee7c3d31ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:44.814604 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e6c4f6-7d15-4911-a5b6-24ee7c3d31ae None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:44.828356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-22e6c4f6-7d15-4911-a5b6-24ee7c3d31ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:44.828356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:44.828356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:44.828356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:44.828356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:44.828356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:44.828356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:44.828356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:44.828356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:44.828356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:44.828356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:44.828356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:44.828356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:44.828356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:44.828356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:44.829295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:44.829295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:44.829295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:44.829295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:44.829295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:44.829295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:44.829295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:44.829295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:44.829295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:44.829295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:44.829295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:44.829295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:44.829295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:44.829295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:44.829295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:44.829295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:44.829295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:44.830233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:44.830233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:44.830233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:44.830233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:44.830233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:44.830233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:44.830233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:44.830233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:44.830233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 301/634] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:44.831433 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c12b1-faca-4b40-81d5-156120bb328e None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.831433 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c12b1-faca-4b40-81d5-156120bb328e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.831698 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c12b1-faca-4b40-81d5-156120bb328e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.851414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-3a6c12b1-faca-4b40-81d5-156120bb328e None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:44.852765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3a6c12b1-faca-4b40-81d5-156120bb328e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:44.864361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ac363-ebc3-49d2-aa66-e9ff74185e2c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:44.899712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0baf0ee-73ae-46d8-b54e-0602ace3f575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:44.900452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baf0ee-73ae-46d8-b54e-0602ace3f575 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.900770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baf0ee-73ae-46d8-b54e-0602ace3f575 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.901020 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baf0ee-73ae-46d8-b54e-0602ace3f575 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.908401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0baf0ee-73ae-46d8-b54e-0602ace3f575 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:44.909858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0baf0ee-73ae-46d8-b54e-0602ace3f575 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:44.910583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:44.924566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f0baf0ee-73ae-46d8-b54e-0602ace3f575 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:44.924566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:44.924566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:44.924566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:44.924566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:44.924566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:44.924566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:44.924566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:44.924566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:44.924566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:44.924566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:44.924566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:44.924566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:44.924566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:44.924566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:44.925452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:44.925452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:44.925452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:44.925452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:44.925452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:44.925452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:44.925452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:44.925452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:44.925452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:44.925452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:44.925452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:44.925452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:44.925452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:44.925452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:44.925452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:44.925452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:44.925452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:44.926318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:44.926318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:44.926318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:44.926318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:44.926318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:44.926318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:44.926318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:44.926318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:44.926739 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 302/635] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:44.926739 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c12b1-faca-4b40-81d5-156120bb328e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:44.926871 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 334/636] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:44.927253 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc6b18-7357-417e-a04e-93f22de55048 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.927530 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc6b18-7357-417e-a04e-93f22de55048 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.927869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc6b18-7357-417e-a04e-93f22de55048 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.929256 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 335/637] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:44.932560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc6b18-7357-417e-a04e-93f22de55048 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:44.999349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:45.031160 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c12b1-faca-4b40-81d5-156120bb328e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:45.037832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc6b18-7357-417e-a04e-93f22de55048 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:45.065350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1001d1-c7b9-421d-beea-270a8e1efe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:45.066135 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1001d1-c7b9-421d-beea-270a8e1efe4e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:45.066413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1001d1-c7b9-421d-beea-270a8e1efe4e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:45.066683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1001d1-c7b9-421d-beea-270a8e1efe4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:45.071369 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cef429-9c86-4620-a538-c5f588429203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:45.072062 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cef429-9c86-4620-a538-c5f588429203 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:45.072347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cef429-9c86-4620-a538-c5f588429203 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:45.072619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cef429-9c86-4620-a538-c5f588429203 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:45.073399 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cef429-9c86-4620-a538-c5f588429203 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:45.073705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cef429-9c86-4620-a538-c5f588429203 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:45.075173 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cef429-9c86-4620-a538-c5f588429203 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:45.076655 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1001d1-c7b9-421d-beea-270a8e1efe4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:45.078088 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1001d1-c7b9-421d-beea-270a8e1efe4e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:45.086694 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 336/638] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:45.087477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cef429-9c86-4620-a538-c5f588429203 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:45.097861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-ad1001d1-c7b9-421d-beea-270a8e1efe4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:45.097861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:45.097861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:45.097861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:45.097861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:45.097861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:45.097861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:45.097861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:45.097861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:45.097861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:45.097861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:45.097861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:45.097861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:45.097861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:45.097861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:45.098878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:45.098878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:45.098878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:45.098878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:45.098878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:45.098878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:45.098878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:45.098878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:45.098878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:45.098878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:45.098878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:45.098878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:45.098878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:45.098878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:45.098878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:45.098878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:45.098878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:45.099789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:45.099789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:45.099789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:45.099789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:45.099789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 303/639] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:45.101324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88739229-a047-4192-bb5f-886774bcaa66 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:45.101639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88739229-a047-4192-bb5f-886774bcaa66 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:45.101886 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88739229-a047-4192-bb5f-886774bcaa66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:45.118214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-88739229-a047-4192-bb5f-886774bcaa66 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:45.119370 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-88739229-a047-4192-bb5f-886774bcaa66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:45.154002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:45.191909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 304/640] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:45.192760 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88739229-a047-4192-bb5f-886774bcaa66 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:45.193480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cef429-9c86-4620-a538-c5f588429203 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:45.218337 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:45.229869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd85ca-9d8b-40c4-8e7f-eaaa561d2479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:45.230529 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd85ca-9d8b-40c4-8e7f-eaaa561d2479 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:45.230807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd85ca-9d8b-40c4-8e7f-eaaa561d2479 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:45.231075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd85ca-9d8b-40c4-8e7f-eaaa561d2479 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:45.231887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd85ca-9d8b-40c4-8e7f-eaaa561d2479 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:45.232192 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd85ca-9d8b-40c4-8e7f-eaaa561d2479 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:45.233633 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd85ca-9d8b-40c4-8e7f-eaaa561d2479 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:45.245434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 337/641] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:45.246217 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd85ca-9d8b-40c4-8e7f-eaaa561d2479 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:45.296492 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88739229-a047-4192-bb5f-886774bcaa66 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:45.330256 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db05e5d-5b2c-41c5-935e-94a905070d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:45.331061 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db05e5d-5b2c-41c5-935e-94a905070d1d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:45.331350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db05e5d-5b2c-41c5-935e-94a905070d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:45.331642 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db05e5d-5b2c-41c5-935e-94a905070d1d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:45.341031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd85ca-9d8b-40c4-8e7f-eaaa561d2479 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:45.341903 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db05e5d-5b2c-41c5-935e-94a905070d1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:45.349769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-0db05e5d-5b2c-41c5-935e-94a905070d1d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:45.349769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:45.349769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:45.349769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:45.349769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:45.349769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:45.349769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:45.349769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:45.349769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:45.349769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:45.349769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:45.349769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:45.349769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:45.349769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:45.349769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:45.349769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:45.350769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:45.350769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:45.350769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:45.350769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:45.350769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:45.350769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:45.350769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:45.350769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:45.350769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:45.350769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:45.350769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:45.350769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:45.350769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:45.350769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:45.350769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:45.350769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:45.351622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:45.351622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:45.351622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:45.351622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:45.351622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:45.351622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:45.351622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:45.351622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:45.351622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:45.351622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:45.351622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:45.351622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:45.351622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 305/642] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:45.352502 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db05e5d-5b2c-41c5-935e-94a905070d1d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:45.378720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbfe45d-9fdf-4831-9e31-c6db24d83137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:45.379372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfe45d-9fdf-4831-9e31-c6db24d83137 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:45.379681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfe45d-9fdf-4831-9e31-c6db24d83137 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:45.379973 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfe45d-9fdf-4831-9e31-c6db24d83137 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:45.380795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbfe45d-9fdf-4831-9e31-c6db24d83137 None admin] RBAC: Adding query filter params (role.id=fb120deba4284112b12de1a251b7da76, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=8d7fe01ef64f46f6a0e77d5945219b33) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:45.381097 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbfe45d-9fdf-4831-9e31-c6db24d83137 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:45.382526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbfe45d-9fdf-4831-9e31-c6db24d83137 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:45.383016 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:45.395673 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 338/643] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:45 2022] GET /identity/v3/role_assignments?user.id=8d7fe01ef64f46f6a0e77d5945219b33&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=fb120deba4284112b12de1a251b7da76 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:45.395673 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbfe45d-9fdf-4831-9e31-c6db24d83137 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:45.443533 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:45.487292 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db05e5d-5b2c-41c5-935e-94a905070d1d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:45.511173 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbfe45d-9fdf-4831-9e31-c6db24d83137 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:45.529932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ef8a8-75e1-4786-a3f1-9e660078584f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:45.531066 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ef8a8-75e1-4786-a3f1-9e660078584f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:45.531343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ef8a8-75e1-4786-a3f1-9e660078584f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:45.531631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ef8a8-75e1-4786-a3f1-9e660078584f None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:45.532395 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ef8a8-75e1-4786-a3f1-9e660078584f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:45.532728 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ef8a8-75e1-4786-a3f1-9e660078584f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:45.534447 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ef8a8-75e1-4786-a3f1-9e660078584f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:45.548661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 306/644] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:45.549100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ef8a8-75e1-4786-a3f1-9e660078584f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:45.556965 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32822f6-4e31-46f7-91f0-6f060610c824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:45.557338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32822f6-4e31-46f7-91f0-6f060610c824 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:45.558074 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32822f6-4e31-46f7-91f0-6f060610c824 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:45.558074 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32822f6-4e31-46f7-91f0-6f060610c824 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:45.571427 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32822f6-4e31-46f7-91f0-6f060610c824 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:45.579517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-d32822f6-4e31-46f7-91f0-6f060610c824 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:45.579517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:45.579517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:45.579517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:45.579517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:45.579517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:45.579517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:45.579517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:45.579517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:45.579517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:45.579517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:45.579517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:45.579517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:45.579517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:45.579517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:45.579517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:45.580603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:45.580603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:45.580603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:45.580603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:45.580603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:45.580603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:45.580603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:45.580603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:45.580603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:45.580603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:45.580603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:45.580603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:45.580603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:45.580603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:45.580603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:45.580603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:45.581590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:45.581590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:45.581590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:45.581590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:45.581590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:45.581590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:45.581590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:45.581590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:45.581590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:45.581590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:45.581590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:45.581590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:45.581590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 339/645] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:45.582343 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32822f6-4e31-46f7-91f0-6f060610c824 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:45.657187 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:45.668183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ef8a8-75e1-4786-a3f1-9e660078584f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:45.696750 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32822f6-4e31-46f7-91f0-6f060610c824 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:45.705612 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c85068-38c7-4273-88f5-0e042847db1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:45.706299 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:45.707419 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c85068-38c7-4273-88f5-0e042847db1f None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:45.707930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c85068-38c7-4273-88f5-0e042847db1f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:45.708414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c85068-38c7-4273-88f5-0e042847db1f None admin] PATH_INFO: `/v3/projects/dfc29b021d1646819137c89e93f33601/users/f28acb0322114aebafcf62bdf9a346df/roles/fb120deba4284112b12de1a251b7da76` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:45.724860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c85068-38c7-4273-88f5-0e042847db1f None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc29b021d1646819137c89e93f33601, user_id=f28acb0322114aebafcf62bdf9a346df, role_id=fb120deba4284112b12de1a251b7da76)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:45.726854 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c85068-38c7-4273-88f5-0e042847db1f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:45.733003 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acdf84c-5914-416e-a220-58728c6c72d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:45.733725 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acdf84c-5914-416e-a220-58728c6c72d0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:45.734090 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acdf84c-5914-416e-a220-58728c6c72d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:45.734344 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acdf84c-5914-416e-a220-58728c6c72d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:45.735197 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acdf84c-5914-416e-a220-58728c6c72d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:45.735502 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acdf84c-5914-416e-a220-58728c6c72d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:45.737014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acdf84c-5914-416e-a220-58728c6c72d0 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:45.751816 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 307/646] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:03:45 2022] PUT /identity/v3/projects/dfc29b021d1646819137c89e93f33601/users/f28acb0322114aebafcf62bdf9a346df/roles/fb120deba4284112b12de1a251b7da76 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:45.751816 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c85068-38c7-4273-88f5-0e042847db1f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:45.752113 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 340/647] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:45.752113 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acdf84c-5914-416e-a220-58728c6c72d0 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:45.864194 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acdf84c-5914-416e-a220-58728c6c72d0 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:45.883574 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c85068-38c7-4273-88f5-0e042847db1f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:45.913294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6cd77-bbca-4491-bf7b-04f4b1337751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:45.913984 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6cd77-bbca-4491-bf7b-04f4b1337751 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:45.914269 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6cd77-bbca-4491-bf7b-04f4b1337751 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:45.914544 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6cd77-bbca-4491-bf7b-04f4b1337751 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:45.915348 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6cd77-bbca-4491-bf7b-04f4b1337751 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:45.915669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6cd77-bbca-4491-bf7b-04f4b1337751 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:45.917137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6cd77-bbca-4491-bf7b-04f4b1337751 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:45.926858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 341/648] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:45.927773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6cd77-bbca-4491-bf7b-04f4b1337751 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:45.929712 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ce8e1-a3b2-4a8d-b49b-3f849b53d62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:45.930698 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ce8e1-a3b2-4a8d-b49b-3f849b53d62a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:45.931131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ce8e1-a3b2-4a8d-b49b-3f849b53d62a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:45.931555 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ce8e1-a3b2-4a8d-b49b-3f849b53d62a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:45.940124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ce8e1-a3b2-4a8d-b49b-3f849b53d62a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:45.941601 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ce8e1-a3b2-4a8d-b49b-3f849b53d62a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:45.945184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:45.958339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-4e6ce8e1-a3b2-4a8d-b49b-3f849b53d62a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 27 21:03:45.958339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:45.958339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:45.958339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:45.958339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:45.958339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:45.958339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:45.958339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:45.958339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:45.958339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:45.958339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:45.958339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:45.958339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:45.958339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:45.958339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:45.958339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:45.959583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:45.959583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:45.959583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:45.959583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:45.959583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:45.959583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:45.959583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:45.959583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:45.959583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:45.959583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:45.959583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:45.959583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:45.959583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:45.959583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:45.959583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:45.959583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:45.959583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:45.959583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:45.961180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:45.961180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:45.961180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:45.961180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:45.961180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:45.961180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:45.961180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:45.961180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:45.961180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 27 21:03:45.961180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:45.961180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 308/649] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:45.962391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:45.965130 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ce8e1-a3b2-4a8d-b49b-3f849b53d62a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:46.043210 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6cd77-bbca-4491-bf7b-04f4b1337751 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:46.079636 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5330fa23-1032-4256-a8ea-ae594126f83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:46.080608 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330fa23-1032-4256-a8ea-ae594126f83d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:46.081021 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330fa23-1032-4256-a8ea-ae594126f83d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:46.081384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330fa23-1032-4256-a8ea-ae594126f83d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:46.090791 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ce8e1-a3b2-4a8d-b49b-3f849b53d62a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:46.093300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5330fa23-1032-4256-a8ea-ae594126f83d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:46.094787 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5330fa23-1032-4256-a8ea-ae594126f83d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:46.113832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-5330fa23-1032-4256-a8ea-ae594126f83d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:46.113832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:46.113832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:46.113832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:46.113832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:46.113832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:46.113832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:46.113832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:46.113832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:46.113832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:46.113832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:46.113832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:46.113832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:46.113832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:46.113832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:46.114808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:46.114808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:46.114808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:46.114808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:46.114808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:46.114808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:46.114808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:46.114808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:46.114808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:46.114808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:46.114808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:46.114808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:46.114808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:46.114808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:46.114808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:46.114808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:46.114808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:46.116627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:46.116627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:46.116627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:46.116627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:46.116627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 342/650] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:46.116627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5330fa23-1032-4256-a8ea-ae594126f83d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:46.131455 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ee083-07c4-43e9-847f-d85c98142f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:46.132397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ee083-07c4-43e9-847f-d85c98142f83 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:46.132789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ee083-07c4-43e9-847f-d85c98142f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:46.133066 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ee083-07c4-43e9-847f-d85c98142f83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:46.144463 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ee083-07c4-43e9-847f-d85c98142f83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:46.146019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ee083-07c4-43e9-847f-d85c98142f83 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:46.164883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-da3ee083-07c4-43e9-847f-d85c98142f83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:46.164883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:46.164883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:46.164883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:46.164883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:46.164883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:46.164883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:46.164883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:46.164883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:46.164883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:46.164883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:46.164883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:46.164883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:46.164883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:46.164883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:46.167004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:46.167004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:46.167004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:46.167004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:46.167004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:46.167004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:46.167004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:46.167004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:46.167004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:46.167004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:46.167004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:46.167004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:46.167004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:46.167004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:46.167004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:46.167004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:46.167004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:46.169235 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:46.169235 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:46.169235 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:46.169235 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:46.169235 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 309/651] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:46.169235 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ee083-07c4-43e9-847f-d85c98142f83 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:46.207771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:46.225031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:46.226930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5330fa23-1032-4256-a8ea-ae594126f83d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:46.264681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f5435-0e2b-4483-8d41-6bc974c008db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:46.265734 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f5435-0e2b-4483-8d41-6bc974c008db None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:46.266162 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f5435-0e2b-4483-8d41-6bc974c008db None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:46.266577 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f5435-0e2b-4483-8d41-6bc974c008db None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:46.267768 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f5435-0e2b-4483-8d41-6bc974c008db None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:46.268348 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f5435-0e2b-4483-8d41-6bc974c008db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:46.269988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f5435-0e2b-4483-8d41-6bc974c008db None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:46.274282 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ee083-07c4-43e9-847f-d85c98142f83 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:46.289870 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 343/652] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Jun 27 21:03:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:46.290891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f5435-0e2b-4483-8d41-6bc974c008db None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:46.314940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d9c22-9d77-4849-be8c-5b993a2446cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:46.315637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d9c22-9d77-4849-be8c-5b993a2446cf None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:46.315927 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d9c22-9d77-4849-be8c-5b993a2446cf None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:46.316201 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d9c22-9d77-4849-be8c-5b993a2446cf None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:46.316949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d9c22-9d77-4849-be8c-5b993a2446cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:46.317370 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d9c22-9d77-4849-be8c-5b993a2446cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:46.318840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d9c22-9d77-4849-be8c-5b993a2446cf None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:46.335424 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 310/653] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:46.336348 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d9c22-9d77-4849-be8c-5b993a2446cf None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:46.397915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f5435-0e2b-4483-8d41-6bc974c008db None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:46.435101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057bf66-e955-4c8d-b53e-857f14a015be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:46.436079 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057bf66-e955-4c8d-b53e-857f14a015be None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:46.436511 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057bf66-e955-4c8d-b53e-857f14a015be None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:46.436976 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057bf66-e955-4c8d-b53e-857f14a015be None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:46.437934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0057bf66-e955-4c8d-b53e-857f14a015be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:46.438385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0057bf66-e955-4c8d-b53e-857f14a015be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:46.440040 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0057bf66-e955-4c8d-b53e-857f14a015be None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:46.440312 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d9c22-9d77-4849-be8c-5b993a2446cf None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:46.452998 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 344/654] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:46.454047 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:46.455450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057bf66-e955-4c8d-b53e-857f14a015be None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:46.458486 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:46.478627 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac38fca-0c74-49aa-9e17-2843540c8a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:46.479288 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac38fca-0c74-49aa-9e17-2843540c8a45 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:46.479568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac38fca-0c74-49aa-9e17-2843540c8a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:46.479872 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac38fca-0c74-49aa-9e17-2843540c8a45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:46.552259 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac38fca-0c74-49aa-9e17-2843540c8a45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:46.553837 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac38fca-0c74-49aa-9e17-2843540c8a45 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:46.565104 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057bf66-e955-4c8d-b53e-857f14a015be None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:46.601524 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c615d-ac3d-43e1-91fa-1626cd9b7159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:46.602743 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c615d-ac3d-43e1-91fa-1626cd9b7159 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:46.603238 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c615d-ac3d-43e1-91fa-1626cd9b7159 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:46.603784 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c615d-ac3d-43e1-91fa-1626cd9b7159 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:46.612479 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c615d-ac3d-43e1-91fa-1626cd9b7159 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:46.614184 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c615d-ac3d-43e1-91fa-1626cd9b7159 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:46.628934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-b39c615d-ac3d-43e1-91fa-1626cd9b7159 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 21:03:46.628934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:46.628934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:46.628934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:46.628934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:46.628934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:46.628934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:46.628934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:46.628934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:46.628934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:46.628934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:46.628934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:46.628934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:46.628934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:46.628934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:46.628934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:46.629915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:46.629915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:46.629915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:46.629915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:46.629915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:46.629915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:46.629915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:46.629915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:46.629915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:46.629915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:46.629915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:46.629915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:46.629915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:46.629915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:46.629915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:46.629915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:46.629915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:46.629915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:46.630826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:46.630826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:46.630826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:46.630826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:46.630826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:46.630826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:46.630826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:46.630826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:46.630826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 21:03:46.630826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:46.633345 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 345/655] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:46.634322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c615d-ac3d-43e1-91fa-1626cd9b7159 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:46.669333 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 311/656] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:46.670304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac38fca-0c74-49aa-9e17-2843540c8a45 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:46.685127 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:46.700242 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:46.740295 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c615d-ac3d-43e1-91fa-1626cd9b7159 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:46.775194 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4dbce-6ca8-41bf-b84f-0d524d6ee32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:46.776349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4dbce-6ca8-41bf-b84f-0d524d6ee32a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:46.776804 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4dbce-6ca8-41bf-b84f-0d524d6ee32a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:46.777246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4dbce-6ca8-41bf-b84f-0d524d6ee32a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:46.781418 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac38fca-0c74-49aa-9e17-2843540c8a45 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:46.786632 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4dbce-6ca8-41bf-b84f-0d524d6ee32a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:46.788457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4dbce-6ca8-41bf-b84f-0d524d6ee32a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:46.803072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-91e4dbce-6ca8-41bf-b84f-0d524d6ee32a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 21:03:46.803072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:46.803072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:46.803072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:46.803072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:46.803072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:46.803072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:46.803072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:46.803072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:46.803072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:46.803072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:46.803072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:46.803072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:46.803072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:46.803072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:46.803072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:46.804036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:46.804036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:46.804036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:46.804036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:46.804036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:46.804036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:46.804036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:46.804036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:46.804036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:46.804036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:46.804036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:46.804036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:46.804036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:46.804036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:46.804036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:46.804036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:46.804036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:46.804036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:46.804950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:46.804950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:46.804950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:46.804950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:46.804950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:46.804950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:46.804950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:46.804950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:46.804950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 21:03:46.804950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:46.807481 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 346/657] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:46.808571 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4dbce-6ca8-41bf-b84f-0d524d6ee32a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:46.817189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57862d36-89a6-47c5-a5e5-ee78f94d876c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:46.817870 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57862d36-89a6-47c5-a5e5-ee78f94d876c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:46.818503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57862d36-89a6-47c5-a5e5-ee78f94d876c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:46.818503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57862d36-89a6-47c5-a5e5-ee78f94d876c None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:46.819369 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57862d36-89a6-47c5-a5e5-ee78f94d876c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:46.819703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57862d36-89a6-47c5-a5e5-ee78f94d876c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:46.821278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57862d36-89a6-47c5-a5e5-ee78f94d876c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:46.847734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 312/658] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Jun 27 21:03:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:46.849052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57862d36-89a6-47c5-a5e5-ee78f94d876c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:46.915906 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4dbce-6ca8-41bf-b84f-0d524d6ee32a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:46.923797 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:46.940683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:46.954219 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef48df-a85e-41dc-af6b-d79174d315cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:46.955323 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef48df-a85e-41dc-af6b-d79174d315cd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:46.955778 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef48df-a85e-41dc-af6b-d79174d315cd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:46.956205 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef48df-a85e-41dc-af6b-d79174d315cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:46.956648 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57862d36-89a6-47c5-a5e5-ee78f94d876c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:46.967781 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef48df-a85e-41dc-af6b-d79174d315cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:46.969601 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef48df-a85e-41dc-af6b-d79174d315cd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:46.988621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-83ef48df-a85e-41dc-af6b-d79174d315cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:46.988621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:46.988621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:46.988621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:46.988621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:46.988621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:46.988621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:46.988621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:46.988621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:46.988621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:46.988621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:46.988621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:46.988621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:46.988621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:46.988621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:46.989736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:46.989736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:46.989736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:46.989736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:46.989736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:46.989736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:46.989736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:46.989736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:46.989736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:46.989736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:46.989736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:46.989736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:46.989736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:46.989736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:46.989736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:46.989736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:46.989736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:46.990774 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:46.990774 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:46.990774 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:46.990774 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:46.992718 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 347/659] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:46.993640 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef48df-a85e-41dc-af6b-d79174d315cd None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:47.007285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ca756-936c-4ea0-80d1-dbe998cc36de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:47.008005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ca756-936c-4ea0-80d1-dbe998cc36de None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:47.008289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ca756-936c-4ea0-80d1-dbe998cc36de None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:47.008563 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ca756-936c-4ea0-80d1-dbe998cc36de None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:47.009400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ca756-936c-4ea0-80d1-dbe998cc36de None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:47.009708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ca756-936c-4ea0-80d1-dbe998cc36de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:47.011140 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ca756-936c-4ea0-80d1-dbe998cc36de None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:47.029471 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 313/660] 10.222.0.27 () {56 vars in 1139 bytes} [Mon Jun 27 21:03:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:47.030370 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ca756-936c-4ea0-80d1-dbe998cc36de None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:47.099376 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef48df-a85e-41dc-af6b-d79174d315cd None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:47.131541 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ca756-936c-4ea0-80d1-dbe998cc36de None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:47.134648 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1638ca-549f-4052-9fda-baac4416e420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:47.135510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1638ca-549f-4052-9fda-baac4416e420 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:47.135949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1638ca-549f-4052-9fda-baac4416e420 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:47.136363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1638ca-549f-4052-9fda-baac4416e420 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:47.159800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:47.169320 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624f76b-c017-44b8-a8b9-341bdf805c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:47.170410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624f76b-c017-44b8-a8b9-341bdf805c37 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:47.170594 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624f76b-c017-44b8-a8b9-341bdf805c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:47.170869 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624f76b-c017-44b8-a8b9-341bdf805c37 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:47.171705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624f76b-c017-44b8-a8b9-341bdf805c37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:47.171933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624f76b-c017-44b8-a8b9-341bdf805c37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:47.173415 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624f76b-c017-44b8-a8b9-341bdf805c37 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:47.184349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:47.192832 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 314/661] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:47.193773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624f76b-c017-44b8-a8b9-341bdf805c37 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:47.213838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1638ca-549f-4052-9fda-baac4416e420 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:47.215539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1638ca-549f-4052-9fda-baac4416e420 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:47.300269 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624f76b-c017-44b8-a8b9-341bdf805c37 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:47.318263 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 348/662] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:47.319361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1638ca-549f-4052-9fda-baac4416e420 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:47.348586 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40896f6f-a0d6-4ab0-9022-f1115547bbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:47.349245 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40896f6f-a0d6-4ab0-9022-f1115547bbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:47.349652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40896f6f-a0d6-4ab0-9022-f1115547bbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:47.349950 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40896f6f-a0d6-4ab0-9022-f1115547bbf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:47.408722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:47.423405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:47.439397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40896f6f-a0d6-4ab0-9022-f1115547bbf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:47.441101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40896f6f-a0d6-4ab0-9022-f1115547bbf5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:47.441719 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1638ca-549f-4052-9fda-baac4416e420 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:47.486410 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a62291-4207-4041-82c4-5268274599a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:47.487532 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a62291-4207-4041-82c4-5268274599a1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:47.487865 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a62291-4207-4041-82c4-5268274599a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:47.488551 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a62291-4207-4041-82c4-5268274599a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:47.498620 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a62291-4207-4041-82c4-5268274599a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:47.500131 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a62291-4207-4041-82c4-5268274599a1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:47.520498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-20a62291-4207-4041-82c4-5268274599a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:47.520498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:47.520498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:47.520498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:47.520498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:47.520498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:47.520498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:47.520498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:47.520498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:47.520498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:47.520498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:47.520498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:47.520498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:47.520498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:47.520498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:47.521457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:47.521457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:47.521457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:47.521457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:47.521457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:47.521457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:47.521457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:47.521457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:47.521457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:47.521457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:47.521457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:47.521457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:47.521457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:47.521457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:47.521457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:47.521457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:47.521457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:47.522333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:47.522333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:47.522333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:47.522333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:47.522333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:47.522333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:47.522333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:47.522333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:47.524954 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 349/663] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:47.524954 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a62291-4207-4041-82c4-5268274599a1 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:47.559421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 315/664] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:47.560734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9d00b-cc7b-448d-bd44-6aac7abe0656 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:47.561018 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9d00b-cc7b-448d-bd44-6aac7abe0656 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:47.561294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9d00b-cc7b-448d-bd44-6aac7abe0656 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:47.566512 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 316/665] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:47.566512 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9d00b-cc7b-448d-bd44-6aac7abe0656 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:47.638130 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a62291-4207-4041-82c4-5268274599a1 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:47.656102 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:47.670740 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9d00b-cc7b-448d-bd44-6aac7abe0656 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:47.673438 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c1aca-5882-4e43-9352-9da11c0b7cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:47.674246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c1aca-5882-4e43-9352-9da11c0b7cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:47.674527 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c1aca-5882-4e43-9352-9da11c0b7cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:47.674801 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c1aca-5882-4e43-9352-9da11c0b7cc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:47.676767 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:47.686440 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c1aca-5882-4e43-9352-9da11c0b7cc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:47.687956 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c1aca-5882-4e43-9352-9da11c0b7cc4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:47.705394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-5d9c1aca-5882-4e43-9352-9da11c0b7cc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:47.705394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:47.705394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:47.705394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:47.705394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:47.705394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:47.705394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:47.705394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:47.705394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:47.705394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:47.705394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:47.705394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:47.705394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:47.705394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:47.705394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:47.706509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:47.706509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:47.706509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:47.706509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:47.706509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:47.706509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:47.706509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:47.706509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:47.706509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:47.706509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:47.706509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:47.706509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:47.706509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:47.706509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:47.706509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:47.706509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:47.706509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:47.707387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:47.707387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:47.707387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:47.707387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:47.707387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 350/666] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:47.707387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c1aca-5882-4e43-9352-9da11c0b7cc4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:47.712589 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84e93c-29c8-4f80-9d73-b0d9a6fa4685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:47.713264 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84e93c-29c8-4f80-9d73-b0d9a6fa4685 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:47.713544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84e93c-29c8-4f80-9d73-b0d9a6fa4685 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:47.713842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84e93c-29c8-4f80-9d73-b0d9a6fa4685 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:47.714730 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b84e93c-29c8-4f80-9d73-b0d9a6fa4685 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:47.715041 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b84e93c-29c8-4f80-9d73-b0d9a6fa4685 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:47.716475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b84e93c-29c8-4f80-9d73-b0d9a6fa4685 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:47.729066 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 317/667] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:47.730288 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6bbd7-d00c-4c1a-b3a6-728369af2135 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:47.730570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6bbd7-d00c-4c1a-b3a6-728369af2135 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:47.730843 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6bbd7-d00c-4c1a-b3a6-728369af2135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:47.748046 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ecd6bbd7-d00c-4c1a-b3a6-728369af2135 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:47.749232 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ecd6bbd7-d00c-4c1a-b3a6-728369af2135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:47.820063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c1aca-5882-4e43-9352-9da11c0b7cc4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:47.823228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 318/668] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:47.824327 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6bbd7-d00c-4c1a-b3a6-728369af2135 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:47.858317 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2ea16-4cf3-4a75-822a-f83e9376bb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:47.859124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2ea16-4cf3-4a75-822a-f83e9376bb75 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:47.859404 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2ea16-4cf3-4a75-822a-f83e9376bb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:47.859707 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2ea16-4cf3-4a75-822a-f83e9376bb75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:47.870177 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2ea16-4cf3-4a75-822a-f83e9376bb75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:47.871669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2ea16-4cf3-4a75-822a-f83e9376bb75 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:47.888430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-72a2ea16-4cf3-4a75-822a-f83e9376bb75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:47.888430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:47.888430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:47.888430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:47.888430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:47.888430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:47.888430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:47.888430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:47.888430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:47.888430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:47.888430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:47.888430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:47.888430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:47.888430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:47.888430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:47.889372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:47.889372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:47.889372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:47.889372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:47.889372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:47.889372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:47.889372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:47.889372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:47.889372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:47.889372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:47.889372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:47.889372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:47.889372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:47.889372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:47.889372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:47.889372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:47.889372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:47.890218 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:47.890218 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:47.890218 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:47.890218 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:47.890218 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 351/669] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:47.890492 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2ea16-4cf3-4a75-822a-f83e9376bb75 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:47.911208 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:47.932794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:47.933310 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6bbd7-d00c-4c1a-b3a6-728369af2135 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:47.968425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cca063-ab26-4831-acae-6901b76d0e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:47.969090 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cca063-ab26-4831-acae-6901b76d0e73 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:47.969369 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cca063-ab26-4831-acae-6901b76d0e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:47.969641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cca063-ab26-4831-acae-6901b76d0e73 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:47.970365 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cca063-ab26-4831-acae-6901b76d0e73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:47.970666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cca063-ab26-4831-acae-6901b76d0e73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:47.972094 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cca063-ab26-4831-acae-6901b76d0e73 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:47.983843 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 319/670] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:47.985134 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761258d-9f42-4cbc-8300-121e60a8f980 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:47.985417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761258d-9f42-4cbc-8300-121e60a8f980 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:47.985696 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761258d-9f42-4cbc-8300-121e60a8f980 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:47.987090 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 320/671] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:47.987623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761258d-9f42-4cbc-8300-121e60a8f980 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:48.002615 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2ea16-4cf3-4a75-822a-f83e9376bb75 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:48.036174 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33852e4f-8fa8-4204-9648-4ca7dabb2ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:48.036834 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33852e4f-8fa8-4204-9648-4ca7dabb2ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:48.037116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33852e4f-8fa8-4204-9648-4ca7dabb2ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:48.037393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33852e4f-8fa8-4204-9648-4ca7dabb2ef6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:48.038217 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33852e4f-8fa8-4204-9648-4ca7dabb2ef6 None admin] RBAC: Adding query filter params (role.id=fb120deba4284112b12de1a251b7da76, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=b43db1fd0b494a3f8588b498b585f247) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:48.038530 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33852e4f-8fa8-4204-9648-4ca7dabb2ef6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:48.040001 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33852e4f-8fa8-4204-9648-4ca7dabb2ef6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:48.045717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 352/672] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:47 2022] GET /identity/v3/role_assignments?user.id=b43db1fd0b494a3f8588b498b585f247&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=fb120deba4284112b12de1a251b7da76 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:48.048560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33852e4f-8fa8-4204-9648-4ca7dabb2ef6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:48.099953 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761258d-9f42-4cbc-8300-121e60a8f980 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:48.139578 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0f990-3f7e-45ea-a5c6-e41af2fd7a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:48.140428 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0f990-3f7e-45ea-a5c6-e41af2fd7a7c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:48.140713 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0f990-3f7e-45ea-a5c6-e41af2fd7a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:48.140990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0f990-3f7e-45ea-a5c6-e41af2fd7a7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:48.149279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:48.155945 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0f990-3f7e-45ea-a5c6-e41af2fd7a7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:48.164736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-fdf0f990-3f7e-45ea-a5c6-e41af2fd7a7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:48.164736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:48.164736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:48.164736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:48.164736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:48.164736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:48.164736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:48.164736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:48.164736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:48.164736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:48.164736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:48.164736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:48.164736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:48.164736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:48.164736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:48.164736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:48.165794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:48.165794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:48.165794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:48.165794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:48.165794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:48.165794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:48.165794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:48.165794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:48.165794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:48.165794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:48.165794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:48.165794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:48.165794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:48.165794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:48.165794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:48.165794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:48.166726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:48.166726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:48.166726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:48.166726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:48.166726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:48.166726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:48.166726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:48.166726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:48.166726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:48.166726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:48.166726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:48.166726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:48.166726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 321/673] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:48.168213 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0f990-3f7e-45ea-a5c6-e41af2fd7a7c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:48.180136 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33852e4f-8fa8-4204-9648-4ca7dabb2ef6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:48.182468 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:48.222925 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb615f3-637f-4264-8d68-5859b305a1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:48.223638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb615f3-637f-4264-8d68-5859b305a1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:48.223944 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb615f3-637f-4264-8d68-5859b305a1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:48.224229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb615f3-637f-4264-8d68-5859b305a1b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:48.224963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb615f3-637f-4264-8d68-5859b305a1b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:48.225277 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb615f3-637f-4264-8d68-5859b305a1b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:48.226690 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb615f3-637f-4264-8d68-5859b305a1b5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:48.242045 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 353/674] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:48.243520 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a39165-37b3-46ee-863f-2a69ecf5ddd0 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:48.243840 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a39165-37b3-46ee-863f-2a69ecf5ddd0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:48.244117 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a39165-37b3-46ee-863f-2a69ecf5ddd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:48.261130 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-91a39165-37b3-46ee-863f-2a69ecf5ddd0 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:48.262340 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-91a39165-37b3-46ee-863f-2a69ecf5ddd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:48.289884 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0f990-3f7e-45ea-a5c6-e41af2fd7a7c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:48.326742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5922620-6dfd-4282-8b6a-382e5a71fc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:48.327830 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5922620-6dfd-4282-8b6a-382e5a71fc5e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:48.328248 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5922620-6dfd-4282-8b6a-382e5a71fc5e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:48.328655 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5922620-6dfd-4282-8b6a-382e5a71fc5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:48.330349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 354/675] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:48.331193 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a39165-37b3-46ee-863f-2a69ecf5ddd0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:48.344210 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5922620-6dfd-4282-8b6a-382e5a71fc5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:48.346049 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5922620-6dfd-4282-8b6a-382e5a71fc5e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:48.361611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-e5922620-6dfd-4282-8b6a-382e5a71fc5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:48.361611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:48.361611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:48.361611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:48.361611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:48.361611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:48.361611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:48.361611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:48.361611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:48.361611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:48.361611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:48.361611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:48.361611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:48.361611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:48.361611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:48.362700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:48.362700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:48.362700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:48.362700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:48.362700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:48.362700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:48.362700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:48.362700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:48.362700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:48.362700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:48.362700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:48.362700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:48.362700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:48.362700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:48.362700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:48.362700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:48.362700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:48.363786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:48.363786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:48.363786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:48.363786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:48.363786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:48.363786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:48.363786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:48.363786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:48.365593 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 322/676] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:48.366547 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5922620-6dfd-4282-8b6a-382e5a71fc5e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:48.430377 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:48.434514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:48.444672 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a39165-37b3-46ee-863f-2a69ecf5ddd0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:48.475451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5922620-6dfd-4282-8b6a-382e5a71fc5e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:48.480536 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d57a4-a265-40f4-bcb4-ef50a2b0381a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:48.481208 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d57a4-a265-40f4-bcb4-ef50a2b0381a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:48.481484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d57a4-a265-40f4-bcb4-ef50a2b0381a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:48.481844 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d57a4-a265-40f4-bcb4-ef50a2b0381a None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:48.482598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d57a4-a265-40f4-bcb4-ef50a2b0381a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:48.482908 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d57a4-a265-40f4-bcb4-ef50a2b0381a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:48.484443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d57a4-a265-40f4-bcb4-ef50a2b0381a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:48.497608 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 355/677] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:48.498480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d57a4-a265-40f4-bcb4-ef50a2b0381a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:48.511161 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee2a76-b536-4c2b-9fcf-acf0b3e05e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:48.512036 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee2a76-b536-4c2b-9fcf-acf0b3e05e27 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:48.512287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee2a76-b536-4c2b-9fcf-acf0b3e05e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:48.512556 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee2a76-b536-4c2b-9fcf-acf0b3e05e27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:48.519978 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee2a76-b536-4c2b-9fcf-acf0b3e05e27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:48.521422 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee2a76-b536-4c2b-9fcf-acf0b3e05e27 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:48.536520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-3bee2a76-b536-4c2b-9fcf-acf0b3e05e27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:48.536520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:48.536520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:48.536520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:48.536520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:48.536520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:48.536520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:48.536520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:48.536520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:48.536520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:48.536520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:48.536520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:48.536520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:48.536520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:48.536520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:48.537615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:48.537615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:48.537615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:48.537615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:48.537615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:48.537615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:48.537615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:48.537615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:48.537615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:48.537615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:48.537615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:48.537615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:48.537615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:48.537615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:48.537615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:48.537615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:48.537615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:48.538707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:48.538707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:48.538707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:48.538707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:48.538707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:48.538707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:48.538707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:48.538707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:48.539951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 323/678] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:48.541955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee2a76-b536-4c2b-9fcf-acf0b3e05e27 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:48.609190 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d57a4-a265-40f4-bcb4-ef50a2b0381a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:48.645263 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911790c-f9be-4273-a873-bb030a8c2bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:48.646124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911790c-f9be-4273-a873-bb030a8c2bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:48.646447 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911790c-f9be-4273-a873-bb030a8c2bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:48.646785 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911790c-f9be-4273-a873-bb030a8c2bd3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:48.654593 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee2a76-b536-4c2b-9fcf-acf0b3e05e27 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:48.659974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911790c-f9be-4273-a873-bb030a8c2bd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:48.670188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-c911790c-f9be-4273-a873-bb030a8c2bd3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:48.670188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:48.670188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:48.670188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:48.670188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:48.670188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:48.670188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:48.670188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:48.670188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:48.670188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:48.670188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:48.670188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:48.670188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:48.670188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:48.670188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:48.670188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:48.671063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:48.671063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:48.671063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:48.671063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:48.671063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:48.671063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:48.671063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:48.671063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:48.671063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:48.671063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:48.671063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:48.671063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:48.671063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:48.671063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:48.671063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:48.671063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:48.672264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:48.672264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:48.672264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:48.672264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:48.672264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:48.672264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:48.672264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:48.672264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:48.672264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:48.672264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:48.672264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:48.672264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:48.673783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:48.677499 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 356/679] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:48.678264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911790c-f9be-4273-a873-bb030a8c2bd3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:48.692268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f32705-a2d2-4f5e-abf1-ee7add4741aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:48.693029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f32705-a2d2-4f5e-abf1-ee7add4741aa None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:48.693233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f32705-a2d2-4f5e-abf1-ee7add4741aa None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:48.693506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f32705-a2d2-4f5e-abf1-ee7add4741aa None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:48.694300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f32705-a2d2-4f5e-abf1-ee7add4741aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:48.694607 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f32705-a2d2-4f5e-abf1-ee7add4741aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:48.696049 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:48.696162 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f32705-a2d2-4f5e-abf1-ee7add4741aa None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:48.708334 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 324/680] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:48.709443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f32705-a2d2-4f5e-abf1-ee7add4741aa None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:48.790839 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911790c-f9be-4273-a873-bb030a8c2bd3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:48.822066 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f32705-a2d2-4f5e-abf1-ee7add4741aa None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:48.828344 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c57c54-12d2-4a55-9656-d2ba5586d630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:48.829152 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c57c54-12d2-4a55-9656-d2ba5586d630 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:48.829429 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c57c54-12d2-4a55-9656-d2ba5586d630 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:48.829752 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c57c54-12d2-4a55-9656-d2ba5586d630 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:48.841675 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c57c54-12d2-4a55-9656-d2ba5586d630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:48.843468 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c57c54-12d2-4a55-9656-d2ba5586d630 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:48.859400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c384ccd-ef26-426d-96bb-feb992c27308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:48.860157 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c384ccd-ef26-426d-96bb-feb992c27308 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:48.860394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-97c57c54-12d2-4a55-9656-d2ba5586d630 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:48.860394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:48.860394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:48.860394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:48.860394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:48.860394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:48.860394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:48.860394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:48.860394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:48.860394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:48.860394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:48.860394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:48.860394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:48.860394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:48.860394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:48.861398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:48.861398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:48.861398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:48.861398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:48.861398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:48.861398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:48.861398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:48.861398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:48.861398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:48.861398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:48.861398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:48.861398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:48.861398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:48.861398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:48.861398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:48.861398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:48.861398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:48.862259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:48.862259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:48.862259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:48.862259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:48.862472 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c384ccd-ef26-426d-96bb-feb992c27308 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:48.862472 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c384ccd-ef26-426d-96bb-feb992c27308 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:48.862472 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c384ccd-ef26-426d-96bb-feb992c27308 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:48.862472 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c384ccd-ef26-426d-96bb-feb992c27308 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:48.863593 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c384ccd-ef26-426d-96bb-feb992c27308 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:48.864070 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 357/681] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:48.865316 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c57c54-12d2-4a55-9656-d2ba5586d630 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:48.878161 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 325/682] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:48.878983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c384ccd-ef26-426d-96bb-feb992c27308 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:48.924006 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:48.934640 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:48.970422 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c57c54-12d2-4a55-9656-d2ba5586d630 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:49.004549 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eeeca6-3866-4dad-8105-4d8ec68ab959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:49.005222 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eeeca6-3866-4dad-8105-4d8ec68ab959 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:49.005502 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eeeca6-3866-4dad-8105-4d8ec68ab959 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:49.005728 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c384ccd-ef26-426d-96bb-feb992c27308 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:49.005833 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eeeca6-3866-4dad-8105-4d8ec68ab959 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:49.006508 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eeeca6-3866-4dad-8105-4d8ec68ab959 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:49.006814 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eeeca6-3866-4dad-8105-4d8ec68ab959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:49.008225 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eeeca6-3866-4dad-8105-4d8ec68ab959 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:49.016174 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 358/683] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:49.017033 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eeeca6-3866-4dad-8105-4d8ec68ab959 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:49.047808 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d220e-354c-4ec8-b981-920c203f9456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:49.048494 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d220e-354c-4ec8-b981-920c203f9456 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:49.048772 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d220e-354c-4ec8-b981-920c203f9456 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:49.049041 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d220e-354c-4ec8-b981-920c203f9456 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:49.049878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d220e-354c-4ec8-b981-920c203f9456 None admin] RBAC: Adding query filter params (role.id=fb120deba4284112b12de1a251b7da76, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=16fda36445d9415aae4df74f31af0601) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:49.050168 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d220e-354c-4ec8-b981-920c203f9456 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:49.051624 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d220e-354c-4ec8-b981-920c203f9456 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:49.067657 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 326/684] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:48 2022] GET /identity/v3/role_assignments?user.id=16fda36445d9415aae4df74f31af0601&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=fb120deba4284112b12de1a251b7da76 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:49.067657 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d220e-354c-4ec8-b981-920c203f9456 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:49.156862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eeeca6-3866-4dad-8105-4d8ec68ab959 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:49.180565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d220e-354c-4ec8-b981-920c203f9456 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:49.182410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:49.200049 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:49.210491 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee8ac3-fe14-4def-b4d5-3f0d3a8e1268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:49.212145 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee8ac3-fe14-4def-b4d5-3f0d3a8e1268 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:49.212848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee8ac3-fe14-4def-b4d5-3f0d3a8e1268 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:49.213331 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee8ac3-fe14-4def-b4d5-3f0d3a8e1268 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:49.214280 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee8ac3-fe14-4def-b4d5-3f0d3a8e1268 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:49.214979 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee8ac3-fe14-4def-b4d5-3f0d3a8e1268 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:49.217441 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee8ac3-fe14-4def-b4d5-3f0d3a8e1268 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:49.224159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc6841-6205-471a-ac02-8af60c34cb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:49.224983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc6841-6205-471a-ac02-8af60c34cb38 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:49.225361 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc6841-6205-471a-ac02-8af60c34cb38 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:49.225732 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc6841-6205-471a-ac02-8af60c34cb38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:49.226679 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc6841-6205-471a-ac02-8af60c34cb38 None admin] RBAC: Adding query filter params (role.id=fb120deba4284112b12de1a251b7da76, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=9c7ce29db6b14176a4c9be436bcddc6e) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:49.227141 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc6841-6205-471a-ac02-8af60c34cb38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:49.228772 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc6841-6205-471a-ac02-8af60c34cb38 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:49.239371 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 359/685] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:49.240333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee8ac3-fe14-4def-b4d5-3f0d3a8e1268 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:49.244008 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 327/686] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:49 2022] GET /identity/v3/role_assignments?user.id=9c7ce29db6b14176a4c9be436bcddc6e&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=fb120deba4284112b12de1a251b7da76 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:49.244996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc6841-6205-471a-ac02-8af60c34cb38 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:49.381903 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc6841-6205-471a-ac02-8af60c34cb38 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:49.387775 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee8ac3-fe14-4def-b4d5-3f0d3a8e1268 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:49.425563 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397ca17-05a7-46f7-b7ff-5e4f5942ecae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:49.426737 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397ca17-05a7-46f7-b7ff-5e4f5942ecae None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:49.427231 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397ca17-05a7-46f7-b7ff-5e4f5942ecae None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:49.427760 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397ca17-05a7-46f7-b7ff-5e4f5942ecae None admin] PATH_INFO: `/v3/users/nova` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:49.437622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397ca17-05a7-46f7-b7ff-5e4f5942ecae None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:49.438697 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef76a97-b5d7-43e8-b875-634a1cdd8d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:49.439143 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397ca17-05a7-46f7-b7ff-5e4f5942ecae None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:49.440200 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef76a97-b5d7-43e8-b875-634a1cdd8d25 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:49.440692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef76a97-b5d7-43e8-b875-634a1cdd8d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:49.441164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef76a97-b5d7-43e8-b875-634a1cdd8d25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:49.453290 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef76a97-b5d7-43e8-b875-634a1cdd8d25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:49.455149 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef76a97-b5d7-43e8-b875-634a1cdd8d25 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:49.456842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-a397ca17-05a7-46f7-b7ff-5e4f5942ecae None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:49.456842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:49.456842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:49.456842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:49.456842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:49.456842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:49.456842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:49.456842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:49.456842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:49.456842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:49.456842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:49.456842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:49.456842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:49.456842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:49.456842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:49.456842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:49.457769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:49.457769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:49.457769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:49.457769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:49.457769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:49.457769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:49.457769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:49.457769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:49.457769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:49.457769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:49.457769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:49.457769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:49.457769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:49.457769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:49.457769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:49.457769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:49.457769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:49.457769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:49.458711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:49.458711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:49.458711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:49.458711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:49.458711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:49.458711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:49.458711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:49.458711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:49.458711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:49.458711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:49.458711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 328/687] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Jun 27 21:03:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:49.463330 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397ca17-05a7-46f7-b7ff-5e4f5942ecae None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:49.477215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-bef76a97-b5d7-43e8-b875-634a1cdd8d25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:49.477215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:49.477215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:49.477215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:49.477215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:49.477215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:49.477215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:49.477215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:49.477215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:49.477215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:49.477215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:49.477215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:49.477215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:49.477215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:49.477215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:49.478128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:49.478128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:49.478128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:49.478128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:49.478128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:49.478128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:49.478128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:49.478128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:49.478128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:49.478128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:49.478128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:49.478128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:49.478128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:49.478128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:49.478128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:49.478128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:49.478128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:49.479011 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:49.479011 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:49.479011 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:49.479011 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:49.481580 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 360/688] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:49.488563 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef76a97-b5d7-43e8-b875-634a1cdd8d25 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:49.496357 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:49.539366 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:49.591342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397ca17-05a7-46f7-b7ff-5e4f5942ecae None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:49.606362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef76a97-b5d7-43e8-b875-634a1cdd8d25 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:49.629058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34609cbc-43d7-454c-a81f-363a7fad4836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:49.630067 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34609cbc-43d7-454c-a81f-363a7fad4836 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:49.630554 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34609cbc-43d7-454c-a81f-363a7fad4836 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:49.630967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34609cbc-43d7-454c-a81f-363a7fad4836 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:49.632002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34609cbc-43d7-454c-a81f-363a7fad4836 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:49.632464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34609cbc-43d7-454c-a81f-363a7fad4836 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:49.634282 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34609cbc-43d7-454c-a81f-363a7fad4836 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:49.647236 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886a868-7ce5-4195-b6cb-cc3680197df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:49.648093 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886a868-7ce5-4195-b6cb-cc3680197df3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:49.648473 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886a868-7ce5-4195-b6cb-cc3680197df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:49.648846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886a868-7ce5-4195-b6cb-cc3680197df3 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:49.649683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886a868-7ce5-4195-b6cb-cc3680197df3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:49.650103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886a868-7ce5-4195-b6cb-cc3680197df3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:49.652788 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886a868-7ce5-4195-b6cb-cc3680197df3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:49.656219 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 329/689] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Jun 27 21:03:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:49.661718 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34609cbc-43d7-454c-a81f-363a7fad4836 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:49.671668 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 361/690] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:49.684602 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886a868-7ce5-4195-b6cb-cc3680197df3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:49.759572 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:49.778449 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34609cbc-43d7-454c-a81f-363a7fad4836 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:49.824189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17ad9f-b1d7-4b13-8eb0-9bc246b58bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:49.825246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17ad9f-b1d7-4b13-8eb0-9bc246b58bac None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:49.825756 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17ad9f-b1d7-4b13-8eb0-9bc246b58bac None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:49.826226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17ad9f-b1d7-4b13-8eb0-9bc246b58bac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:49.828830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886a868-7ce5-4195-b6cb-cc3680197df3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:49.852135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:49.867277 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bb1b8-f3d2-486e-a1ea-62f0c4e1ad9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:49.868133 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bb1b8-f3d2-486e-a1ea-62f0c4e1ad9a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:49.868415 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bb1b8-f3d2-486e-a1ea-62f0c4e1ad9a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:49.868687 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bb1b8-f3d2-486e-a1ea-62f0c4e1ad9a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:49.879633 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564bb1b8-f3d2-486e-a1ea-62f0c4e1ad9a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:49.879633 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564bb1b8-f3d2-486e-a1ea-62f0c4e1ad9a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:49.893646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-564bb1b8-f3d2-486e-a1ea-62f0c4e1ad9a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:03:49.893646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:49.893646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:49.893646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:49.893646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:49.893646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:49.893646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:49.893646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:49.893646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:49.893646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:49.893646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:49.893646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:49.893646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:49.893646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:49.893646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:49.893646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:49.894654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:49.894654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:49.894654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:49.894654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:49.894654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:49.894654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:49.894654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:49.894654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:49.894654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:49.894654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:49.894654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:49.894654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:49.894654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:49.894654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:49.894654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:49.894654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:49.894654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:49.894654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:49.895573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:49.895573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:49.895573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:49.895573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:49.895573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:49.895573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:49.895573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:49.895573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:49.895573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:03:49.895573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:49.895573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 362/691] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:49.902578 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bb1b8-f3d2-486e-a1ea-62f0c4e1ad9a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:49.999641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17ad9f-b1d7-4b13-8eb0-9bc246b58bac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:49.999641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17ad9f-b1d7-4b13-8eb0-9bc246b58bac None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:50.079765 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bb1b8-f3d2-486e-a1ea-62f0c4e1ad9a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:50.112210 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3209f-1837-45fb-8c7a-20a22d7d4506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:50.119635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3209f-1837-45fb-8c7a-20a22d7d4506 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:50.119635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3209f-1837-45fb-8c7a-20a22d7d4506 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:50.119635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3209f-1837-45fb-8c7a-20a22d7d4506 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:50.119635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3209f-1837-45fb-8c7a-20a22d7d4506 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:50.119635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3209f-1837-45fb-8c7a-20a22d7d4506 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:50.119635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3209f-1837-45fb-8c7a-20a22d7d4506 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:50.139083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 363/692] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Jun 27 21:03:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:50.140040 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3209f-1837-45fb-8c7a-20a22d7d4506 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:50.151628 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:50.172004 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:50.173987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 330/693] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 21:03:50.179012 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17ad9f-b1d7-4b13-8eb0-9bc246b58bac None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:50.247646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3209f-1837-45fb-8c7a-20a22d7d4506 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:50.285059 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f714b-c201-4fe2-9546-5cef8e2022a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:50.285768 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f714b-c201-4fe2-9546-5cef8e2022a9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:50.286051 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f714b-c201-4fe2-9546-5cef8e2022a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:50.286325 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f714b-c201-4fe2-9546-5cef8e2022a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:50.293984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17ad9f-b1d7-4b13-8eb0-9bc246b58bac None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:50.330206 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e64860-6022-4aa9-b0ae-007721b2a6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:50.331023 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e64860-6022-4aa9-b0ae-007721b2a6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:50.331310 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e64860-6022-4aa9-b0ae-007721b2a6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:50.331583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e64860-6022-4aa9-b0ae-007721b2a6e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:50.342581 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e64860-6022-4aa9-b0ae-007721b2a6e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:50.344070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e64860-6022-4aa9-b0ae-007721b2a6e2 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:50.359180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f714b-c201-4fe2-9546-5cef8e2022a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:50.360793 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f714b-c201-4fe2-9546-5cef8e2022a9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:50.363100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-05e64860-6022-4aa9-b0ae-007721b2a6e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:50.363100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:50.363100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:50.363100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:50.363100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:50.363100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:50.363100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:50.363100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:50.363100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:50.363100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:50.363100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:50.363100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:50.363100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:50.363100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:50.363100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:50.363981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:50.363981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:50.363981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:50.363981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:50.363981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:50.363981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:50.363981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:50.363981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:50.363981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:50.363981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:50.363981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:50.363981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:50.363981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:50.363981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:50.363981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:50.363981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:50.363981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:50.364862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:50.364862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:50.364862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:50.364862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:50.369289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 331/694] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:50.370630 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0a9ef-1013-45e6-a7e0-34212fd31b54 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:50.371022 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0a9ef-1013-45e6-a7e0-34212fd31b54 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:50.371309 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0a9ef-1013-45e6-a7e0-34212fd31b54 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:50.372963 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 332/695] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:50.373806 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0a9ef-1013-45e6-a7e0-34212fd31b54 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:50.380106 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:50.415726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:50.477787 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 364/696] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 21:03:50.479486 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3d371-1d4e-48d5-8b39-2993bae17b88 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:50.479937 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3d371-1d4e-48d5-8b39-2993bae17b88 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:50.480449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3d371-1d4e-48d5-8b39-2993bae17b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:50.492740 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0a9ef-1013-45e6-a7e0-34212fd31b54 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:50.498558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f2d3d371-1d4e-48d5-8b39-2993bae17b88 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:50.499785 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f2d3d371-1d4e-48d5-8b39-2993bae17b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:50.534702 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73253df8-568c-4e12-b0c4-67db4db1115b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:50.536163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73253df8-568c-4e12-b0c4-67db4db1115b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:50.536707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73253df8-568c-4e12-b0c4-67db4db1115b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:50.537258 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73253df8-568c-4e12-b0c4-67db4db1115b None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:50.538305 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73253df8-568c-4e12-b0c4-67db4db1115b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:50.538818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73253df8-568c-4e12-b0c4-67db4db1115b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:50.541252 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73253df8-568c-4e12-b0c4-67db4db1115b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:50.555205 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 333/697] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:50.556292 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73253df8-568c-4e12-b0c4-67db4db1115b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:50.583196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 365/698] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:50.584358 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3d371-1d4e-48d5-8b39-2993bae17b88 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:50.637719 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:50.655833 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:50.663109 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73253df8-568c-4e12-b0c4-67db4db1115b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:50.697212 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3d371-1d4e-48d5-8b39-2993bae17b88 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:50.697567 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f1428-e89f-4329-9cf5-6989410f8def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:50.698449 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f1428-e89f-4329-9cf5-6989410f8def None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:50.698800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f1428-e89f-4329-9cf5-6989410f8def None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:50.699139 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f1428-e89f-4329-9cf5-6989410f8def None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:50.710124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f1428-e89f-4329-9cf5-6989410f8def None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:50.711671 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f1428-e89f-4329-9cf5-6989410f8def None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:50.729033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-4d1f1428-e89f-4329-9cf5-6989410f8def None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:50.729033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:50.729033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:50.729033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:50.729033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:50.729033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:50.729033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:50.729033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:50.729033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:50.729033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:50.729033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:50.729033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:50.729033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:50.729033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:50.729033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:50.730159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:50.730159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:50.730159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:50.730159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:50.730159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:50.730159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:50.730159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:50.730159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:50.730159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:50.730159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:50.730159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:50.730159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:50.730159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:50.730159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:50.730159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:50.730159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:50.730159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:50.731241 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:50.731241 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:50.731241 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:50.731241 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:50.736756 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 334/699] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:50.737653 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f1428-e89f-4329-9cf5-6989410f8def None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:50.738529 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414cef0-6311-4632-aa96-cf152d3bfa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:50.739511 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414cef0-6311-4632-aa96-cf152d3bfa42 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:50.739972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414cef0-6311-4632-aa96-cf152d3bfa42 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:50.740488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414cef0-6311-4632-aa96-cf152d3bfa42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:50.748179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414cef0-6311-4632-aa96-cf152d3bfa42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:50.750034 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414cef0-6311-4632-aa96-cf152d3bfa42 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:50.765832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4414cef0-6311-4632-aa96-cf152d3bfa42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:50.765832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:50.765832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:50.765832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:50.765832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:50.765832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:50.765832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:50.765832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:50.765832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:50.765832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:50.765832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:50.765832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:50.765832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:50.765832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:50.765832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:50.766722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:50.766722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:50.766722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:50.766722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:50.766722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:50.766722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:50.766722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:50.766722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:50.766722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:50.766722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:50.766722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:50.766722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:50.766722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:50.766722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:50.766722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:50.766722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:50.766722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:50.767684 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:50.767684 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:50.767684 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:50.767684 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:50.767684 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:50.767684 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:50.767684 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:50.767684 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:50.770335 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 366/700] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:50.771484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414cef0-6311-4632-aa96-cf152d3bfa42 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:50.845762 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f1428-e89f-4329-9cf5-6989410f8def None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:50.881059 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece8883-d585-4d45-88fd-6b75a17366ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:50.881866 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece8883-d585-4d45-88fd-6b75a17366ab None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:50.882146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece8883-d585-4d45-88fd-6b75a17366ab None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:50.882419 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece8883-d585-4d45-88fd-6b75a17366ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:50.886146 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414cef0-6311-4632-aa96-cf152d3bfa42 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:50.894872 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece8883-d585-4d45-88fd-6b75a17366ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:50.896173 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece8883-d585-4d45-88fd-6b75a17366ab None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:50.897480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:50.905094 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:50.917074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-0ece8883-d585-4d45-88fd-6b75a17366ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:50.917074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:50.917074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:50.917074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:50.917074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:50.917074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:50.917074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:50.917074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:50.917074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:50.917074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:50.917074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:50.917074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:50.917074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:50.917074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:50.917074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:50.917989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:50.917989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:50.917989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:50.917989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:50.917989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:50.917989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:50.917989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:50.917989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:50.917989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:50.917989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:50.917989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:50.917989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:50.917989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:50.917989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:50.917989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:50.917989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:50.917989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:50.919628 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:50.919628 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:50.919628 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:50.919628 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:50.919628 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 335/701] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:50.919628 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece8883-d585-4d45-88fd-6b75a17366ab None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:50.931695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee746e61-584f-44d2-98fa-67fddba73239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:50.932776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee746e61-584f-44d2-98fa-67fddba73239 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:50.933058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee746e61-584f-44d2-98fa-67fddba73239 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:50.933338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee746e61-584f-44d2-98fa-67fddba73239 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:50.934060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee746e61-584f-44d2-98fa-67fddba73239 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:50.934367 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee746e61-584f-44d2-98fa-67fddba73239 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:50.936150 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee746e61-584f-44d2-98fa-67fddba73239 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:50.955495 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 367/702] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:50.957351 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee746e61-584f-44d2-98fa-67fddba73239 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:51.067847 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece8883-d585-4d45-88fd-6b75a17366ab None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:51.071808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee746e61-584f-44d2-98fa-67fddba73239 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:51.107741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae38354-6c88-41d9-8d85-7147b920363b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:51.108337 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae38354-6c88-41d9-8d85-7147b920363b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:51.108614 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae38354-6c88-41d9-8d85-7147b920363b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:51.108875 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae38354-6c88-41d9-8d85-7147b920363b None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:51.109597 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae38354-6c88-41d9-8d85-7147b920363b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:51.109897 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae38354-6c88-41d9-8d85-7147b920363b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:51.111488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae38354-6c88-41d9-8d85-7147b920363b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:51.116208 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391dbc9-43d6-48b6-8cec-3d25bf3dc49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:51.117147 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391dbc9-43d6-48b6-8cec-3d25bf3dc49d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:51.117675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391dbc9-43d6-48b6-8cec-3d25bf3dc49d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:51.118640 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391dbc9-43d6-48b6-8cec-3d25bf3dc49d None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:51.119703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391dbc9-43d6-48b6-8cec-3d25bf3dc49d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:51.120764 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391dbc9-43d6-48b6-8cec-3d25bf3dc49d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:51.122682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391dbc9-43d6-48b6-8cec-3d25bf3dc49d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:51.123449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 368/703] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:51.124403 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae38354-6c88-41d9-8d85-7147b920363b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:51.139927 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 336/704] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:51.141114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391dbc9-43d6-48b6-8cec-3d25bf3dc49d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:51.158700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:51.217335 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:51.234828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae38354-6c88-41d9-8d85-7147b920363b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:51.269570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d1364-87db-4101-9b41-f1b08e85cbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:51.270398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d1364-87db-4101-9b41-f1b08e85cbf1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:51.270676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d1364-87db-4101-9b41-f1b08e85cbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:51.270947 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d1364-87db-4101-9b41-f1b08e85cbf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:51.272651 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391dbc9-43d6-48b6-8cec-3d25bf3dc49d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:51.278031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d1364-87db-4101-9b41-f1b08e85cbf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:51.279502 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d1364-87db-4101-9b41-f1b08e85cbf1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:51.294759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-879d1364-87db-4101-9b41-f1b08e85cbf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:51.294759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:51.294759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:51.294759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:51.294759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:51.294759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:51.294759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:51.294759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:51.294759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:51.294759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:51.294759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:51.294759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:51.294759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:51.294759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:51.294759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:51.295695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:51.295695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:51.295695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:51.295695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:51.295695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:51.295695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:51.295695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:51.295695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:51.295695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:51.295695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:51.295695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:51.295695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:51.295695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:51.295695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:51.295695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:51.295695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:51.295695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:51.296573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:51.296573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:51.296573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:51.296573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:51.296573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:51.296573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:51.296573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:51.296573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:51.299961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 369/705] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:51.301048 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d1364-87db-4101-9b41-f1b08e85cbf1 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:51.318047 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6a6cb-d4f5-4566-9de9-40a4fbea5b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:51.318778 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6a6cb-d4f5-4566-9de9-40a4fbea5b57 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:51.318998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6a6cb-d4f5-4566-9de9-40a4fbea5b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:51.319272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6a6cb-d4f5-4566-9de9-40a4fbea5b57 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:51.320007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6a6cb-d4f5-4566-9de9-40a4fbea5b57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:51.321570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6a6cb-d4f5-4566-9de9-40a4fbea5b57 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:51.344350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 337/706] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Jun 27 21:03:51 2022] POST /identity/v3/users => generated 254 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:51.346101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6a6cb-d4f5-4566-9de9-40a4fbea5b57 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:51.408507 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:51.423757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d1364-87db-4101-9b41-f1b08e85cbf1 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:51.463466 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7c9bb-b525-479c-84af-124f93f6134c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:51.464308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7c9bb-b525-479c-84af-124f93f6134c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:51.464692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7c9bb-b525-479c-84af-124f93f6134c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:51.465055 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7c9bb-b525-479c-84af-124f93f6134c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:51.466111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7c9bb-b525-479c-84af-124f93f6134c None admin] RBAC: Adding query filter params (role.id=5c3c095bb53943eca95c4e049cd8cbf5, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=8d7fe01ef64f46f6a0e77d5945219b33) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:51.466612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7c9bb-b525-479c-84af-124f93f6134c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:51.468248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7c9bb-b525-479c-84af-124f93f6134c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:51.469753 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6a6cb-d4f5-4566-9de9-40a4fbea5b57 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:51.480928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 370/707] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:51 2022] GET /identity/v3/role_assignments?user.id=8d7fe01ef64f46f6a0e77d5945219b33&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=5c3c095bb53943eca95c4e049cd8cbf5 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:51.480928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b92f8-65d9-4d66-975a-47f727bb7945 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:51.480928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b92f8-65d9-4d66-975a-47f727bb7945 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:51.480928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b92f8-65d9-4d66-975a-47f727bb7945 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:51.480928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 371/708] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:51.484865 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fa52f-a4bb-4186-bf7a-bc1bef4973e1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:51.485326 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fa52f-a4bb-4186-bf7a-bc1bef4973e1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:51.485736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fa52f-a4bb-4186-bf7a-bc1bef4973e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:51.516430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:51.523172 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8f8fa52f-a4bb-4186-bf7a-bc1bef4973e1 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:51.524721 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8f8fa52f-a4bb-4186-bf7a-bc1bef4973e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:51.542101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188ce54-a146-4764-b9a6-0a3fafbd513f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:51.544933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188ce54-a146-4764-b9a6-0a3fafbd513f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:51.545222 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188ce54-a146-4764-b9a6-0a3fafbd513f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:51.545506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188ce54-a146-4764-b9a6-0a3fafbd513f None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:51.546333 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188ce54-a146-4764-b9a6-0a3fafbd513f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:51.546654 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188ce54-a146-4764-b9a6-0a3fafbd513f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:51.548309 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188ce54-a146-4764-b9a6-0a3fafbd513f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:51.565777 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 338/709] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:51.571848 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188ce54-a146-4764-b9a6-0a3fafbd513f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:51.624015 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 372/710] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:03:51.641328 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fa52f-a4bb-4186-bf7a-bc1bef4973e1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:51.708568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188ce54-a146-4764-b9a6-0a3fafbd513f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:51.740075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:51.751352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e3b5a-2402-44f5-b287-e06116a4bba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:51.752330 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e3b5a-2402-44f5-b287-e06116a4bba8 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:51.752823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e3b5a-2402-44f5-b287-e06116a4bba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:51.753280 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e3b5a-2402-44f5-b287-e06116a4bba8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:51.754307 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e3b5a-2402-44f5-b287-e06116a4bba8 None admin] RBAC: Adding query filter params (role.id=fb120deba4284112b12de1a251b7da76, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=f28acb0322114aebafcf62bdf9a346df) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:51.754800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e3b5a-2402-44f5-b287-e06116a4bba8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:51.756466 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e3b5a-2402-44f5-b287-e06116a4bba8 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:51.761513 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fa52f-a4bb-4186-bf7a-bc1bef4973e1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:51.763630 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 339/711] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:51 2022] GET /identity/v3/role_assignments?user.id=f28acb0322114aebafcf62bdf9a346df&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=fb120deba4284112b12de1a251b7da76 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:51.765086 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f808fe-90d8-4076-95e9-231126745d6a None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:51.765520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f808fe-90d8-4076-95e9-231126745d6a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:51.765942 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f808fe-90d8-4076-95e9-231126745d6a None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:51.771798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 340/712] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:51.773313 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1cefc-4c99-4190-b572-42109c43de8b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:51.773751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1cefc-4c99-4190-b572-42109c43de8b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:51.774189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1cefc-4c99-4190-b572-42109c43de8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:51.801890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfc012-35ef-42ff-961e-34cd58231730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:51.803186 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfc012-35ef-42ff-961e-34cd58231730 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:51.803738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfc012-35ef-42ff-961e-34cd58231730 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:51.804223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfc012-35ef-42ff-961e-34cd58231730 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:51.805139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedfc012-35ef-42ff-961e-34cd58231730 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:51.807130 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedfc012-35ef-42ff-961e-34cd58231730 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:51.812732 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-0ee1cefc-4c99-4190-b572-42109c43de8b None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:51.814066 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0ee1cefc-4c99-4190-b572-42109c43de8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:51.825164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-dedfc012-35ef-42ff-961e-34cd58231730 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 21:03:51.825164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:51.825164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:51.825164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:51.825164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:51.825164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:51.825164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:51.825164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:51.825164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 21:03:51.825164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:51.825164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:51.825164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:51.825164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:51.825164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:51.825164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:51.825164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:51.826306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:51.826306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:51.826306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:51.826306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:51.826306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:51.826306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:51.826306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:51.826306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:51.826306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:51.826306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:51.826306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:51.826306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:51.826306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:51.826306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:51.826306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:51.826306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:51.827126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:51.827126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:51.827126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:51.827126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:51.827126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:51.827126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:51.827126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:51.827126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:51.827126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:51.827126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:51.827126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:51.827126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:51.827126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:51.827126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 21:03:51.827126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:51.830952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 373/713] 10.222.0.27 () {56 vars in 1129 bytes} [Mon Jun 27 21:03:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:51.837990 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfc012-35ef-42ff-961e-34cd58231730 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:51.917565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:51.974720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfc012-35ef-42ff-961e-34cd58231730 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:51.986614 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 341/714] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:52.020520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1cefc-4c99-4190-b572-42109c43de8b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:52.023906 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2fc23-af40-4072-a598-2c5ea545fbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:52.024603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2fc23-af40-4072-a598-2c5ea545fbd7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:52.024879 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2fc23-af40-4072-a598-2c5ea545fbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:52.025153 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2fc23-af40-4072-a598-2c5ea545fbd7 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:52.025870 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2fc23-af40-4072-a598-2c5ea545fbd7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:52.026177 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2fc23-af40-4072-a598-2c5ea545fbd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:52.027577 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2fc23-af40-4072-a598-2c5ea545fbd7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:52.032748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 374/715] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Jun 27 21:03:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:52.041589 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2fc23-af40-4072-a598-2c5ea545fbd7 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:52.059667 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:52.164457 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1cefc-4c99-4190-b572-42109c43de8b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:52.169209 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2fc23-af40-4072-a598-2c5ea545fbd7 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:52.206534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48134ec5-f73e-4790-90bb-d9aa37df03ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:52.207989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48134ec5-f73e-4790-90bb-d9aa37df03ab None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:52.208518 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48134ec5-f73e-4790-90bb-d9aa37df03ab None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:52.209012 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48134ec5-f73e-4790-90bb-d9aa37df03ab None admin] PATH_INFO: `/v3/services/network` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:52.209887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ac6d9-a615-4024-b579-73753a075481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:52.210077 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48134ec5-f73e-4790-90bb-d9aa37df03ab None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:52.210614 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ac6d9-a615-4024-b579-73753a075481 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:52.210901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ac6d9-a615-4024-b579-73753a075481 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:52.211178 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ac6d9-a615-4024-b579-73753a075481 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:52.211997 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ac6d9-a615-4024-b579-73753a075481 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:52.212225 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48134ec5-f73e-4790-90bb-d9aa37df03ab None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:52.212331 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ac6d9-a615-4024-b579-73753a075481 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:52.213774 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ac6d9-a615-4024-b579-73753a075481 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:52.221327 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 375/716] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Jun 27 21:03:52 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:52.232705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-48134ec5-f73e-4790-90bb-d9aa37df03ab None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 21:03:52.232705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:52.232705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:52.232705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:52.232705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:52.232705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:52.232705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:52.232705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:52.232705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 21:03:52.232705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:52.232705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:52.232705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:52.232705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:52.232705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:52.232705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:52.232705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:52.232705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:52.233965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:52.233965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:52.233965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:52.233965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:52.233965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:52.233965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:52.233965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:52.233965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:52.233965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:52.233965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:52.233965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:52.233965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:52.233965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:52.233965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:52.233965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:52.233965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:52.234861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:52.234861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:52.234861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:52.234861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:52.234861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:52.234861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:52.234861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:52.234861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:52.234861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:52.234861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:52.234861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:52.234861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:52.234861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 21:03:52.234861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:52.234861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 342/717] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:52 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:52.240089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ac6d9-a615-4024-b579-73753a075481 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:52.330674 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:52.359088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ac6d9-a615-4024-b579-73753a075481 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:52.398552 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb7556-03e8-4de6-87be-aef199a6ebd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:52.399352 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb7556-03e8-4de6-87be-aef199a6ebd1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:52.399735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb7556-03e8-4de6-87be-aef199a6ebd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:52.400080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb7556-03e8-4de6-87be-aef199a6ebd1 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:52.400896 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb7556-03e8-4de6-87be-aef199a6ebd1 None admin] RBAC: Adding query filter params (name=network) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:52.401275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb7556-03e8-4de6-87be-aef199a6ebd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:52.402795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb7556-03e8-4de6-87be-aef199a6ebd1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:52.408998 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 376/718] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Jun 27 21:03:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:52.414835 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48134ec5-f73e-4790-90bb-d9aa37df03ab None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:52.456510 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:52.549545 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48134ec5-f73e-4790-90bb-d9aa37df03ab None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:52.601472 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73045b44-9510-453c-a153-b08745fb2e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:52.602552 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73045b44-9510-453c-a153-b08745fb2e92 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:52.602973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73045b44-9510-453c-a153-b08745fb2e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:52.603382 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73045b44-9510-453c-a153-b08745fb2e92 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:52.604267 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73045b44-9510-453c-a153-b08745fb2e92 None admin] RBAC: Adding query filter params (type=network) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:52.604730 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73045b44-9510-453c-a153-b08745fb2e92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:52.606361 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73045b44-9510-453c-a153-b08745fb2e92 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:52.616348 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 343/719] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Jun 27 21:03:52 2022] GET /identity/v3/services?type=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:54.046544 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ca374-9cd3-456e-81a8-c5cf0b74a80a None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.047132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ca374-9cd3-456e-81a8-c5cf0b74a80a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.047132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ca374-9cd3-456e-81a8-c5cf0b74a80a None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.048878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 377/720] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:54.054174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585ce7e-642a-411b-8c08-4c38d9e9cadb None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.054580 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585ce7e-642a-411b-8c08-4c38d9e9cadb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.054766 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585ce7e-642a-411b-8c08-4c38d9e9cadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.071770 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d585ce7e-642a-411b-8c08-4c38d9e9cadb None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:54.072975 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d585ce7e-642a-411b-8c08-4c38d9e9cadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:54.111788 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbca6d8-0289-4b3c-84ea-ad7f5e20377e None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.112140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbca6d8-0289-4b3c-84ea-ad7f5e20377e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.112345 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbca6d8-0289-4b3c-84ea-ad7f5e20377e None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.113948 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 378/721] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:54.119158 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9ffa7-0d7f-45b8-8c3b-bedf86468cfe None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.119441 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9ffa7-0d7f-45b8-8c3b-bedf86468cfe None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.119743 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9ffa7-0d7f-45b8-8c3b-bedf86468cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.143636 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c3c9ffa7-0d7f-45b8-8c3b-bedf86468cfe None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:54.144871 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c3c9ffa7-0d7f-45b8-8c3b-bedf86468cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:54.159965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 344/722] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:54.193665 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585ce7e-642a-411b-8c08-4c38d9e9cadb None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:54.238151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:54.292670 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 379/723] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:03:54.321818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585ce7e-642a-411b-8c08-4c38d9e9cadb None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:54.324938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9ffa7-0d7f-45b8-8c3b-bedf86468cfe None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:54.356608 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499db874-bfbc-4b6c-b01b-74cad32c5934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:54.357427 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499db874-bfbc-4b6c-b01b-74cad32c5934 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.357709 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499db874-bfbc-4b6c-b01b-74cad32c5934 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.357985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499db874-bfbc-4b6c-b01b-74cad32c5934 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.368675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499db874-bfbc-4b6c-b01b-74cad32c5934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:54.376979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-499db874-bfbc-4b6c-b01b-74cad32c5934 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:54.376979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:54.376979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:54.376979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:54.376979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:54.376979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:54.376979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:54.376979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:54.376979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:54.376979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:54.376979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:54.376979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:54.376979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:54.376979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:54.376979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:54.376979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:54.377960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:54.377960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:54.377960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:54.377960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:54.377960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:54.377960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:54.377960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:54.377960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:54.377960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:54.377960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:54.377960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:54.377960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:54.377960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:54.377960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:54.377960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:54.377960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:54.378825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:54.378825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:54.378825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:54.378825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:54.378825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:54.378825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:54.378825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:54.378825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:54.378825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:54.378825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:54.378825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:54.378825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:54.383056 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 345/724] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:54.384480 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499db874-bfbc-4b6c-b01b-74cad32c5934 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:54.419976 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:54.477980 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:54.491113 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499db874-bfbc-4b6c-b01b-74cad32c5934 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:54.497695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9ffa7-0d7f-45b8-8c3b-bedf86468cfe None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:54.525464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9e2b1-7bcf-4bed-b436-0f4af38db41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:54.526149 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9e2b1-7bcf-4bed-b436-0f4af38db41f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.526434 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9e2b1-7bcf-4bed-b436-0f4af38db41f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.526710 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9e2b1-7bcf-4bed-b436-0f4af38db41f None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.531650 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b9e2b1-7bcf-4bed-b436-0f4af38db41f None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:54.531650 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b9e2b1-7bcf-4bed-b436-0f4af38db41f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:54.531650 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b9e2b1-7bcf-4bed-b436-0f4af38db41f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:54.542551 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 346/725] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:54.543927 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9e2b1-7bcf-4bed-b436-0f4af38db41f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:54.570139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939cf8b-9f53-49f8-8be8-6e05dc4caec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:54.570956 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939cf8b-9f53-49f8-8be8-6e05dc4caec6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.571245 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939cf8b-9f53-49f8-8be8-6e05dc4caec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.571517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939cf8b-9f53-49f8-8be8-6e05dc4caec6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.582121 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9939cf8b-9f53-49f8-8be8-6e05dc4caec6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:54.589909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-9939cf8b-9f53-49f8-8be8-6e05dc4caec6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:54.589909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:54.589909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:54.589909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:54.589909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:54.589909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:54.589909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:54.589909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:54.589909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:54.589909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:54.589909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:54.589909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:54.589909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:54.589909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:54.589909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:54.589909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:54.590830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:54.590830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:54.590830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:54.590830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:54.590830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:54.590830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:54.590830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:54.590830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:54.590830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:54.590830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:54.590830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:54.590830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:54.590830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:54.590830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:54.590830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:54.590830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:54.591666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:54.591666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:54.591666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:54.591666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:54.591666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:54.591666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:54.591666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:54.591666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:54.591666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:54.591666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:54.591666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:54.591666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:54.591666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 380/726] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:54.597437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29cd59-bab6-49c7-b299-0f5292917f33 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.597718 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29cd59-bab6-49c7-b299-0f5292917f33 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.597995 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29cd59-bab6-49c7-b299-0f5292917f33 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.599562 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 381/727] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:54.603564 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29cd59-bab6-49c7-b299-0f5292917f33 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:54.665731 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9e2b1-7bcf-4bed-b436-0f4af38db41f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:54.705175 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ab3fd-0f6e-4a93-83a0-2d9f662b092a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:54.706085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ab3fd-0f6e-4a93-83a0-2d9f662b092a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.706372 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ab3fd-0f6e-4a93-83a0-2d9f662b092a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.706698 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ab3fd-0f6e-4a93-83a0-2d9f662b092a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.719348 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ab3fd-0f6e-4a93-83a0-2d9f662b092a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:54.720927 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ab3fd-0f6e-4a93-83a0-2d9f662b092a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:54.733259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29cd59-bab6-49c7-b299-0f5292917f33 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:54.741482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-367ab3fd-0f6e-4a93-83a0-2d9f662b092a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:54.741482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:54.741482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:54.741482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:54.741482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:54.741482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:54.741482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:54.741482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:54.741482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:54.741482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:54.741482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:54.741482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:54.741482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:54.741482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:54.741482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:54.742350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:54.742350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:54.742350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:54.742350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:54.742350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:54.742350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:54.742350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:54.742350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:54.742350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:54.742350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:54.742350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:54.742350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:54.742350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:54.742350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:54.742350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:54.742350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:54.742350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:54.743214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:54.743214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:54.743214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:54.743214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:54.743443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 347/728] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:54.747643 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee7184-a923-4f5f-9122-13c2c8984db4 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.747643 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee7184-a923-4f5f-9122-13c2c8984db4 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.747643 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee7184-a923-4f5f-9122-13c2c8984db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.751734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:54.762899 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:54.764713 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-2fee7184-a923-4f5f-9122-13c2c8984db4 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:54.765951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2fee7184-a923-4f5f-9122-13c2c8984db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:54.771968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c763d5-9ca8-4842-959d-c0273dde531d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:54.772647 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c763d5-9ca8-4842-959d-c0273dde531d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.772934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c763d5-9ca8-4842-959d-c0273dde531d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.773214 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c763d5-9ca8-4842-959d-c0273dde531d None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.773968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c763d5-9ca8-4842-959d-c0273dde531d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:54.774279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c763d5-9ca8-4842-959d-c0273dde531d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:54.775699 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c763d5-9ca8-4842-959d-c0273dde531d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:54.790419 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 382/729] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:54.790419 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959aadd-e6a3-48d8-8dc0-8602c7fe6cec None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.790419 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959aadd-e6a3-48d8-8dc0-8602c7fe6cec None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.790419 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959aadd-e6a3-48d8-8dc0-8602c7fe6cec None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.790419 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 383/730] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:54.791415 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f959aadd-e6a3-48d8-8dc0-8602c7fe6cec None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:54.842884 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 348/731] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:54.843892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee7184-a923-4f5f-9122-13c2c8984db4 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:54.914154 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f959aadd-e6a3-48d8-8dc0-8602c7fe6cec None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:54.949959 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679470e7-6924-4bd5-ade9-59967d7ad1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:54.950627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679470e7-6924-4bd5-ade9-59967d7ad1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.950906 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679470e7-6924-4bd5-ade9-59967d7ad1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.951177 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679470e7-6924-4bd5-ade9-59967d7ad1e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.951926 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679470e7-6924-4bd5-ade9-59967d7ad1e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:54.952230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679470e7-6924-4bd5-ade9-59967d7ad1e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:54.953609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679470e7-6924-4bd5-ade9-59967d7ad1e2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:54.958848 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee7184-a923-4f5f-9122-13c2c8984db4 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:54.967940 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 384/732] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:54.970204 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d72431-7c0e-40fc-9237-5c708d166481 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.970805 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d72431-7c0e-40fc-9237-5c708d166481 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.971112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d72431-7c0e-40fc-9237-5c708d166481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.991592 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-23d72431-7c0e-40fc-9237-5c708d166481 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:54.993036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-23d72431-7c0e-40fc-9237-5c708d166481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:54.997679 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:55.003386 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e9fe4-9fe5-4652-b022-394963812655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:55.004278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e9fe4-9fe5-4652-b022-394963812655 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:55.004565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e9fe4-9fe5-4652-b022-394963812655 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:55.005036 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e9fe4-9fe5-4652-b022-394963812655 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:55.016423 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e9fe4-9fe5-4652-b022-394963812655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:55.017765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e9fe4-9fe5-4652-b022-394963812655 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:55.037839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-c27e9fe4-9fe5-4652-b022-394963812655 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:55.037839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:55.037839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:55.037839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:55.037839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:55.037839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:55.037839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:55.037839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:55.037839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:55.037839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:55.037839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:55.037839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:55.037839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:55.037839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:55.037839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:55.038865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:55.038865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:55.038865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:55.038865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:55.038865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:55.038865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:55.038865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:55.038865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:55.038865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:55.038865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:55.038865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:55.038865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:55.038865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:55.038865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:55.038865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:55.038865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:55.038865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:55.039766 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:55.039766 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:55.039766 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:55.039766 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:55.039766 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 349/733] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:55.040465 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e9fe4-9fe5-4652-b022-394963812655 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:55.047709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:55.069642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 385/734] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:55.070495 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d72431-7c0e-40fc-9237-5c708d166481 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:55.161037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e9fe4-9fe5-4652-b022-394963812655 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:55.174809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d72431-7c0e-40fc-9237-5c708d166481 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:55.199292 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886f22f-5529-4d52-8e3b-9377cd1f0c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:55.200128 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886f22f-5529-4d52-8e3b-9377cd1f0c88 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:55.200409 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886f22f-5529-4d52-8e3b-9377cd1f0c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:55.200681 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886f22f-5529-4d52-8e3b-9377cd1f0c88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:55.209369 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617828bc-4e15-404b-aa13-2d32124a4ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:55.210196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617828bc-4e15-404b-aa13-2d32124a4ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:55.210478 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617828bc-4e15-404b-aa13-2d32124a4ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:55.210756 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617828bc-4e15-404b-aa13-2d32124a4ca3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:55.211170 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886f22f-5529-4d52-8e3b-9377cd1f0c88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:55.218313 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617828bc-4e15-404b-aa13-2d32124a4ca3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:55.218873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-2886f22f-5529-4d52-8e3b-9377cd1f0c88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:55.218873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:55.218873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:55.218873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:55.218873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:55.218873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:55.218873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:55.218873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:55.218873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:55.218873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:55.218873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:55.218873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:55.218873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:55.218873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:55.218873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:55.218873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:55.219792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:55.219792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:55.219792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:55.219792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:55.219792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:55.219792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:55.219792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:55.219792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:55.219792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:55.219792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:55.219792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:55.219792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:55.219792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:55.219792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:55.219792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:55.219792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:55.220727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:55.220727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:55.220727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:55.220727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:55.220727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:55.220727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:55.220727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:55.220727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:55.220727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:55.220727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:55.220727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:55.220727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:55.221433 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617828bc-4e15-404b-aa13-2d32124a4ca3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:55.221503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 350/735] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:55.221503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886f22f-5529-4d52-8e3b-9377cd1f0c88 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:55.240338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-617828bc-4e15-404b-aa13-2d32124a4ca3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:03:55.240338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:55.240338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:55.240338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:55.240338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:55.240338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:55.240338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:55.240338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:55.240338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:55.240338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:55.240338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:55.240338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:55.240338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:55.240338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:55.240338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:55.240338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:55.241308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:55.241308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:55.241308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:55.241308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:55.241308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:55.241308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:55.241308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:55.241308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:55.241308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:55.241308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:55.241308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:55.241308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:55.241308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:55.241308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:55.241308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:55.241308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:55.241308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:55.241308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:55.242288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:55.242288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:55.242288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:55.242288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:55.242288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:55.242288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:55.242288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:55.242288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:55.242288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:03:55.242288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:55.242288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 386/736] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Jun 27 21:03:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:55.242876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617828bc-4e15-404b-aa13-2d32124a4ca3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:55.282015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:55.287077 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:55.342756 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886f22f-5529-4d52-8e3b-9377cd1f0c88 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:55.350257 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617828bc-4e15-404b-aa13-2d32124a4ca3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:55.378740 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944d532-d0bf-498e-9180-7b6d41e2fb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:55.379366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944d532-d0bf-498e-9180-7b6d41e2fb50 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:55.379670 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944d532-d0bf-498e-9180-7b6d41e2fb50 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:55.379942 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944d532-d0bf-498e-9180-7b6d41e2fb50 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:55.380670 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944d532-d0bf-498e-9180-7b6d41e2fb50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:55.380977 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944d532-d0bf-498e-9180-7b6d41e2fb50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:55.382352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944d532-d0bf-498e-9180-7b6d41e2fb50 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:55.385223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df3bcd-a0a7-4322-b16d-9ebfc7adb4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:55.385916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df3bcd-a0a7-4322-b16d-9ebfc7adb4d3 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:55.386194 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df3bcd-a0a7-4322-b16d-9ebfc7adb4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:55.386465 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df3bcd-a0a7-4322-b16d-9ebfc7adb4d3 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:55.387183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df3bcd-a0a7-4322-b16d-9ebfc7adb4d3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:55.388608 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df3bcd-a0a7-4322-b16d-9ebfc7adb4d3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:55.393878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 351/737] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:55.394710 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944d532-d0bf-498e-9180-7b6d41e2fb50 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:55.403873 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 387/738] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Jun 27 21:03:55 2022] POST /identity/v3/services => generated 251 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:55.404729 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df3bcd-a0a7-4322-b16d-9ebfc7adb4d3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:55.516446 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944d532-d0bf-498e-9180-7b6d41e2fb50 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:55.521610 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:55.527710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:55.539720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df3bcd-a0a7-4322-b16d-9ebfc7adb4d3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:55.560659 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a556b-f065-43cb-817a-64f1b4889108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:55.561980 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a556b-f065-43cb-817a-64f1b4889108 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:55.561980 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a556b-f065-43cb-817a-64f1b4889108 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:55.561980 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a556b-f065-43cb-817a-64f1b4889108 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:55.562662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a556b-f065-43cb-817a-64f1b4889108 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:55.562972 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a556b-f065-43cb-817a-64f1b4889108 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:55.564438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a556b-f065-43cb-817a-64f1b4889108 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:55.574451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 352/739] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:55.575792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf7b63-89c4-44d1-a4dd-7a1039b5a120 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:55.576079 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf7b63-89c4-44d1-a4dd-7a1039b5a120 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:55.576353 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf7b63-89c4-44d1-a4dd-7a1039b5a120 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:55.577757 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 353/740] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:55.578229 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf7b63-89c4-44d1-a4dd-7a1039b5a120 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:55.582721 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d3cda-7af8-425b-881f-cfa2b8a1dabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:55.583040 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d3cda-7af8-425b-881f-cfa2b8a1dabe None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:55.583324 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d3cda-7af8-425b-881f-cfa2b8a1dabe None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:55.583629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d3cda-7af8-425b-881f-cfa2b8a1dabe None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:55.584706 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d3cda-7af8-425b-881f-cfa2b8a1dabe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:55.584813 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d3cda-7af8-425b-881f-cfa2b8a1dabe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:55.586302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d3cda-7af8-425b-881f-cfa2b8a1dabe None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:55.606064 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 388/741] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Jun 27 21:03:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:55.607219 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d3cda-7af8-425b-881f-cfa2b8a1dabe None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:55.692018 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf7b63-89c4-44d1-a4dd-7a1039b5a120 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:55.722152 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d3cda-7af8-425b-881f-cfa2b8a1dabe None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:55.728908 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc7dd5-38e7-4f31-97fa-ea87de5c9d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:55.729967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc7dd5-38e7-4f31-97fa-ea87de5c9d2d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:55.730882 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc7dd5-38e7-4f31-97fa-ea87de5c9d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:55.731373 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc7dd5-38e7-4f31-97fa-ea87de5c9d2d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:55.740086 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc7dd5-38e7-4f31-97fa-ea87de5c9d2d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:55.742004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc7dd5-38e7-4f31-97fa-ea87de5c9d2d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:55.757037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-32dc7dd5-38e7-4f31-97fa-ea87de5c9d2d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:55.757037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:55.757037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:55.757037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:55.757037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:55.757037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:55.757037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:55.757037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:55.757037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:55.757037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:55.757037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:55.757037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:55.757037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:55.757037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:55.757037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:55.757037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:55.758032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:55.758032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:55.758032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:55.758032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:55.758032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:55.758032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:55.758032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:55.758032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:55.758032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:55.758032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:55.758032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:55.758032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:55.758032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:55.758032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:55.758032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:55.758032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:55.758032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:55.758032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:55.758966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:55.758966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:55.758966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:55.758966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:55.758966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:55.758966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:55.758966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:55.758966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:55.758966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:55.758966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:55.758966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 354/742] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Jun 27 21:03:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:55.759561 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372b387-1177-467c-bd7a-f916449fbde2 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:55.760212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372b387-1177-467c-bd7a-f916449fbde2 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:55.760212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372b387-1177-467c-bd7a-f916449fbde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:55.763636 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca38b3-67d1-4007-9787-87abac07863d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:55.764462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca38b3-67d1-4007-9787-87abac07863d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:55.764750 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca38b3-67d1-4007-9787-87abac07863d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:55.765027 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca38b3-67d1-4007-9787-87abac07863d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:55.775935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca38b3-67d1-4007-9787-87abac07863d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:55.777093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d372b387-1177-467c-bd7a-f916449fbde2 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:55.777351 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca38b3-67d1-4007-9787-87abac07863d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:55.778618 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d372b387-1177-467c-bd7a-f916449fbde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:55.795770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-82ca38b3-67d1-4007-9787-87abac07863d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:55.795770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:55.795770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:55.795770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:55.795770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:55.795770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:55.795770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:55.795770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:55.795770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:55.795770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:55.795770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:55.795770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:55.795770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:55.795770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:55.795770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:55.796856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:55.796856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:55.796856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:55.796856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:55.796856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:55.796856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:55.796856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:55.796856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:55.796856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:55.796856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:55.796856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:55.796856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:55.796856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:55.796856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:55.796856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:55.796856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:55.796856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:55.797949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:55.797949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:55.797949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:55.797949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:55.801802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 389/743] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:55.802646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca38b3-67d1-4007-9787-87abac07863d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:55.803278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:55.812668 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:55.871740 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 355/744] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:55 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:55.872880 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372b387-1177-467c-bd7a-f916449fbde2 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:55.911224 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca38b3-67d1-4007-9787-87abac07863d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:55.946358 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d87042-07dd-4547-a990-879a3ea96f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:55.947029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d87042-07dd-4547-a990-879a3ea96f06 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:55.947395 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d87042-07dd-4547-a990-879a3ea96f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:55.947575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d87042-07dd-4547-a990-879a3ea96f06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:55.984157 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372b387-1177-467c-bd7a-f916449fbde2 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:56.020949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d87042-07dd-4547-a990-879a3ea96f06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:56.022554 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d87042-07dd-4547-a990-879a3ea96f06 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:56.023654 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f49c36-e668-4197-978c-a208b761e414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:56.024352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f49c36-e668-4197-978c-a208b761e414 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:56.024636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f49c36-e668-4197-978c-a208b761e414 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:56.024911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f49c36-e668-4197-978c-a208b761e414 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:56.025746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f49c36-e668-4197-978c-a208b761e414 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:56.026111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f49c36-e668-4197-978c-a208b761e414 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:56.027520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f49c36-e668-4197-978c-a208b761e414 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:56.031213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:56.045842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 356/745] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Jun 27 21:03:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:56.046477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f49c36-e668-4197-978c-a208b761e414 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:56.061772 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:56.136331 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 390/746] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:55 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:56.136331 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d87042-07dd-4547-a990-879a3ea96f06 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:56.154808 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f49c36-e668-4197-978c-a208b761e414 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:56.190704 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3256b-ea1b-4a18-ac37-ac595b7452fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:56.191396 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3256b-ea1b-4a18-ac37-ac595b7452fa None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:56.191721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3256b-ea1b-4a18-ac37-ac595b7452fa None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:56.192006 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3256b-ea1b-4a18-ac37-ac595b7452fa None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:56.192757 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3256b-ea1b-4a18-ac37-ac595b7452fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:56.193076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3256b-ea1b-4a18-ac37-ac595b7452fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:56.194521 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3256b-ea1b-4a18-ac37-ac595b7452fa None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:56.207433 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 357/747] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:56.208262 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3256b-ea1b-4a18-ac37-ac595b7452fa None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:56.236165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d87042-07dd-4547-a990-879a3ea96f06 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:56.244226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:56.272103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d15a7d-da8c-4545-abaf-fc9374911ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:56.272824 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d15a7d-da8c-4545-abaf-fc9374911ed3 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:56.273110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d15a7d-da8c-4545-abaf-fc9374911ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:56.273413 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d15a7d-da8c-4545-abaf-fc9374911ed3 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:56.274127 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d15a7d-da8c-4545-abaf-fc9374911ed3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:56.275506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d15a7d-da8c-4545-abaf-fc9374911ed3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:56.288999 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:56.291039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 391/748] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Jun 27 21:03:56 2022] POST /identity/v3/services => generated 240 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:56.291893 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d15a7d-da8c-4545-abaf-fc9374911ed3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:56.327156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3256b-ea1b-4a18-ac37-ac595b7452fa None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:56.383165 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda4483-1397-4d7b-91b7-80635669849a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:56.384475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda4483-1397-4d7b-91b7-80635669849a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:56.384988 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda4483-1397-4d7b-91b7-80635669849a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:56.385540 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda4483-1397-4d7b-91b7-80635669849a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:56.398670 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda4483-1397-4d7b-91b7-80635669849a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:56.400775 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda4483-1397-4d7b-91b7-80635669849a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:56.401086 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d15a7d-da8c-4545-abaf-fc9374911ed3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:56.422084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-5eda4483-1397-4d7b-91b7-80635669849a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:56.422084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:56.422084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:56.422084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:56.422084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:56.422084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:56.422084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:56.422084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:56.422084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:56.422084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:56.422084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:56.422084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:56.422084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:56.422084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:56.422084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:56.423151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:56.423151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:56.423151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:56.423151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:56.423151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:56.423151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:56.423151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:56.423151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:56.423151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:56.423151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:56.423151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:56.423151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:56.423151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:56.423151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:56.423151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:56.423151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:56.423151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:56.424082 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:56.424082 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:56.424082 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:56.424082 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:56.426585 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 358/749] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:56.427975 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda4483-1397-4d7b-91b7-80635669849a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:56.443547 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df781f99-91ec-4c03-bf2a-4380b714f935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:56.444402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df781f99-91ec-4c03-bf2a-4380b714f935 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:56.444671 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df781f99-91ec-4c03-bf2a-4380b714f935 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:56.444942 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df781f99-91ec-4c03-bf2a-4380b714f935 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:56.457801 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df781f99-91ec-4c03-bf2a-4380b714f935 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:56.459632 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df781f99-91ec-4c03-bf2a-4380b714f935 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:56.477089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-df781f99-91ec-4c03-bf2a-4380b714f935 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:56.477089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:56.477089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:56.477089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:56.477089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:56.477089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:56.477089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:56.477089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:56.477089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:56.477089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:56.477089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:56.477089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:56.477089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:56.477089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:56.477089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:56.478112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:56.478112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:56.478112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:56.478112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:56.478112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:56.478112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:56.478112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:56.478112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:56.478112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:56.478112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:56.478112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:56.478112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:56.478112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:56.478112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:56.478112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:56.478112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:56.478112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:56.478981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:56.478981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:56.478981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:56.478981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:56.483745 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 392/750] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:56.483745 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df781f99-91ec-4c03-bf2a-4380b714f935 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:56.506360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:56.549819 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda4483-1397-4d7b-91b7-80635669849a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:56.575446 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:56.591414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93595d6a-889f-42de-b3a3-5d77d5e97cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:56.592415 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93595d6a-889f-42de-b3a3-5d77d5e97cce None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:56.592801 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93595d6a-889f-42de-b3a3-5d77d5e97cce None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:56.593175 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93595d6a-889f-42de-b3a3-5d77d5e97cce None admin] PATH_INFO: `/v3/users/swift` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:56.603925 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93595d6a-889f-42de-b3a3-5d77d5e97cce None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:56.605511 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93595d6a-889f-42de-b3a3-5d77d5e97cce None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:56.612415 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df781f99-91ec-4c03-bf2a-4380b714f935 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:56.622279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-93595d6a-889f-42de-b3a3-5d77d5e97cce None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:03:56.622279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:56.622279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:56.622279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:56.622279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:56.622279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:56.622279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:56.622279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:56.622279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:56.622279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:56.622279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:56.622279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:56.622279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:56.622279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:56.622279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:56.622279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:56.623226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:56.623226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:56.623226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:56.623226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:56.623226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:56.623226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:56.623226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:56.623226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:56.623226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:56.623226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:56.623226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:56.623226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:56.623226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:56.623226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:56.623226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:56.623226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:56.623226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:56.623226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:56.624566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:56.624566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:56.624566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:56.624566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:56.624566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:56.624566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:56.624566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:56.624566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:56.624566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:03:56.624566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:56.624566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 359/751] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:56.625141 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93595d6a-889f-42de-b3a3-5d77d5e97cce None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:56.667594 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0a8c0-58b1-4227-9c18-a694f9f937f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:56.668928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0a8c0-58b1-4227-9c18-a694f9f937f7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:56.670317 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0a8c0-58b1-4227-9c18-a694f9f937f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:56.670843 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0a8c0-58b1-4227-9c18-a694f9f937f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:56.671810 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c0a8c0-58b1-4227-9c18-a694f9f937f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:56.672341 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c0a8c0-58b1-4227-9c18-a694f9f937f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:56.674058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c0a8c0-58b1-4227-9c18-a694f9f937f7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:56.691291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 393/752] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:56.693052 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0a8c0-58b1-4227-9c18-a694f9f937f7 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:56.750243 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93595d6a-889f-42de-b3a3-5d77d5e97cce None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:56.790010 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6b689-a1fc-4218-aebd-018f5de0996e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:56.790784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6b689-a1fc-4218-aebd-018f5de0996e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:56.791138 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6b689-a1fc-4218-aebd-018f5de0996e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:56.791489 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6b689-a1fc-4218-aebd-018f5de0996e None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:56.792333 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6b689-a1fc-4218-aebd-018f5de0996e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:56.793136 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6b689-a1fc-4218-aebd-018f5de0996e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:56.795242 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:56.797864 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6b689-a1fc-4218-aebd-018f5de0996e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:56.819518 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 360/753] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:56.820434 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6b689-a1fc-4218-aebd-018f5de0996e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:56.822179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0a8c0-58b1-4227-9c18-a694f9f937f7 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:56.861854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab173f-40ac-42fd-ab49-a52feaf5f5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:56.862987 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab173f-40ac-42fd-ab49-a52feaf5f5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:56.863432 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab173f-40ac-42fd-ab49-a52feaf5f5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:56.864029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab173f-40ac-42fd-ab49-a52feaf5f5c6 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:56.865457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab173f-40ac-42fd-ab49-a52feaf5f5c6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:56.865945 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab173f-40ac-42fd-ab49-a52feaf5f5c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:56.867585 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab173f-40ac-42fd-ab49-a52feaf5f5c6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:56.889205 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 394/754] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Jun 27 21:03:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:56.890470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab173f-40ac-42fd-ab49-a52feaf5f5c6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:56.907664 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:56.976764 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6b689-a1fc-4218-aebd-018f5de0996e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:57.014155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab173f-40ac-42fd-ab49-a52feaf5f5c6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:57.057059 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ecbeac-2353-4070-b947-52bbd28cc174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:57.058062 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ecbeac-2353-4070-b947-52bbd28cc174 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.058445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ecbeac-2353-4070-b947-52bbd28cc174 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.058816 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ecbeac-2353-4070-b947-52bbd28cc174 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.061795 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfc1fb-a4e1-4272-90e5-1da005a50170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:57.061940 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:57.067396 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ecbeac-2353-4070-b947-52bbd28cc174 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:57.069034 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ecbeac-2353-4070-b947-52bbd28cc174 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:57.070191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfc1fb-a4e1-4272-90e5-1da005a50170 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.070683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfc1fb-a4e1-4272-90e5-1da005a50170 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.071108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfc1fb-a4e1-4272-90e5-1da005a50170 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-62ecbeac-2353-4070-b947-52bbd28cc174 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:57.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:57.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:57.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:57.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:57.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:57.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:57.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:57.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:57.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:57.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:57.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:57.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:57.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:57.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:57.086744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:57.086744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:57.086744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:57.086744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:57.086744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:57.086744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:57.086744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:57.086744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:57.086744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:57.086744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:57.086744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:57.086744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:57.086744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:57.086744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:57.086744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:57.086744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:57.086744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:57.088628 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:57.088628 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:57.088628 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:57.088628 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:57.088628 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:57.088628 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:57.088628 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:57.088628 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:57.089393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 395/755] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:57.090262 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ecbeac-2353-4070-b947-52bbd28cc174 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:57.103158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdfc1fb-a4e1-4272-90e5-1da005a50170 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:57.104959 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdfc1fb-a4e1-4272-90e5-1da005a50170 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:57.120155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-ecdfc1fb-a4e1-4272-90e5-1da005a50170 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:57.120155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:57.120155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:57.120155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:57.120155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:57.120155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:57.120155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:57.120155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:57.120155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:57.120155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:57.120155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:57.120155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:57.120155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:57.120155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:57.120155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:57.121237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:57.121237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:57.121237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:57.121237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:57.121237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:57.121237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:57.121237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:57.121237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:57.121237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:57.121237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:57.121237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:57.121237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:57.121237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:57.121237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:57.121237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:57.121237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:57.121237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:57.122343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:57.122343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:57.122343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:57.122343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:57.122343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:57.122343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:57.122343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:57.122343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:57.131193 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 361/756] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:57.132386 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfc1fb-a4e1-4272-90e5-1da005a50170 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:57.204666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ecbeac-2353-4070-b947-52bbd28cc174 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:57.240976 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d851c0-9ef1-433e-beb3-f1410bfe9812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:57.241802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d851c0-9ef1-433e-beb3-f1410bfe9812 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.242201 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d851c0-9ef1-433e-beb3-f1410bfe9812 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.242572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d851c0-9ef1-433e-beb3-f1410bfe9812 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.311752 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:57.320277 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d851c0-9ef1-433e-beb3-f1410bfe9812 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:57.321704 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d851c0-9ef1-433e-beb3-f1410bfe9812 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:57.359784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfc1fb-a4e1-4272-90e5-1da005a50170 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:57.362764 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:57.402578 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a45969-a695-4d0e-b983-7072b2752661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:57.403647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a45969-a695-4d0e-b983-7072b2752661 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.404156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a45969-a695-4d0e-b983-7072b2752661 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.404750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a45969-a695-4d0e-b983-7072b2752661 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.405838 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a45969-a695-4d0e-b983-7072b2752661 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:57.406478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a45969-a695-4d0e-b983-7072b2752661 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:57.409449 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a45969-a695-4d0e-b983-7072b2752661 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:57.423831 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 362/757] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:57.424825 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 396/758] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:03:57.425255 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a45969-a695-4d0e-b983-7072b2752661 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:57.429894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d851c0-9ef1-433e-beb3-f1410bfe9812 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:57.532649 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d851c0-9ef1-433e-beb3-f1410bfe9812 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:57.535945 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:57.538170 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a45969-a695-4d0e-b983-7072b2752661 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:57.568568 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fe017-179b-4df6-9ed6-23765f31ec38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:57.569264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fe017-179b-4df6-9ed6-23765f31ec38 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.569548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fe017-179b-4df6-9ed6-23765f31ec38 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.569790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fe017-179b-4df6-9ed6-23765f31ec38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.570690 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fe017-179b-4df6-9ed6-23765f31ec38 None admin] RBAC: Adding query filter params (role.id=fb120deba4284112b12de1a251b7da76, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=e802da0b078e457da466a9a7b0f15085) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:57.570949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fe017-179b-4df6-9ed6-23765f31ec38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:57.572763 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fe017-179b-4df6-9ed6-23765f31ec38 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:57.577785 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34321483-797e-4ed4-8115-93231afd562d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:57.578718 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34321483-797e-4ed4-8115-93231afd562d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.579292 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34321483-797e-4ed4-8115-93231afd562d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.580120 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34321483-797e-4ed4-8115-93231afd562d None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.581200 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34321483-797e-4ed4-8115-93231afd562d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:57.581727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34321483-797e-4ed4-8115-93231afd562d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:57.583454 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34321483-797e-4ed4-8115-93231afd562d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:57.585699 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 397/759] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:57 2022] GET /identity/v3/role_assignments?user.id=e802da0b078e457da466a9a7b0f15085&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=fb120deba4284112b12de1a251b7da76 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:57.586692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fe017-179b-4df6-9ed6-23765f31ec38 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:57.598172 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 363/760] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:57.599970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d3536-b458-4730-b20f-5275cf4baf4f None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.600490 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d3536-b458-4730-b20f-5275cf4baf4f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.601000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d3536-b458-4730-b20f-5275cf4baf4f None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.602973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 364/761] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:57.604044 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d3536-b458-4730-b20f-5275cf4baf4f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:57.632184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:57.704585 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fe017-179b-4df6-9ed6-23765f31ec38 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:57.724121 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d3536-b458-4730-b20f-5275cf4baf4f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:57.743104 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7384c-ca47-4434-8fcb-13204da54aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:57.744056 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7384c-ca47-4434-8fcb-13204da54aca None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.744350 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7384c-ca47-4434-8fcb-13204da54aca None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.744622 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7384c-ca47-4434-8fcb-13204da54aca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.757527 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b7384c-ca47-4434-8fcb-13204da54aca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:57.759005 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b7384c-ca47-4434-8fcb-13204da54aca None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:57.762789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195c2dc-8ca4-4d21-93f3-271533810e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:57.763985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195c2dc-8ca4-4d21-93f3-271533810e31 None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.764474 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195c2dc-8ca4-4d21-93f3-271533810e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.764958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195c2dc-8ca4-4d21-93f3-271533810e31 None admin] PATH_INFO: `/v3/projects/dfc29b021d1646819137c89e93f33601/users/8d7fe01ef64f46f6a0e77d5945219b33/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.767344 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8195c2dc-8ca4-4d21-93f3-271533810e31 None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc29b021d1646819137c89e93f33601, user_id=8d7fe01ef64f46f6a0e77d5945219b33, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:57.769037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8195c2dc-8ca4-4d21-93f3-271533810e31 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:57.781679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-c5b7384c-ca47-4434-8fcb-13204da54aca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:57.781679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:57.781679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:57.781679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:57.781679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:57.781679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:57.781679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:57.781679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:57.781679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:57.781679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:57.781679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:57.781679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:57.781679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:57.781679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:57.781679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:57.782643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:57.782643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:57.782643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:57.782643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:57.782643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:57.782643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:57.782643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:57.782643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:57.782643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:57.782643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:57.782643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:57.782643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:57.782643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:57.782643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:57.782643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:57.782643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:57.782643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:57.784294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:57.784294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:57.784294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:57.784294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:57.784294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 398/762] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:57.785613 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cdb56-f318-4225-999f-122a122cb5d4 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.785949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cdb56-f318-4225-999f-122a122cb5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.786265 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cdb56-f318-4225-999f-122a122cb5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.787015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 365/763] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:03:57 2022] PUT /identity/v3/projects/dfc29b021d1646819137c89e93f33601/users/8d7fe01ef64f46f6a0e77d5945219b33/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:57.802381 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195c2dc-8ca4-4d21-93f3-271533810e31 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:57.806667 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ee3cdb56-f318-4225-999f-122a122cb5d4 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:57.808177 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ee3cdb56-f318-4225-999f-122a122cb5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:57.846568 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:57.890816 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 399/764] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:03:57.914465 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3cdb56-f318-4225-999f-122a122cb5d4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:57.971510 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:57.995659 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195c2dc-8ca4-4d21-93f3-271533810e31 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:58.056937 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5b639-ac77-494a-b3d9-08e4e19475db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:58.057618 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b639-ac77-494a-b3d9-08e4e19475db None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:58.057896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b639-ac77-494a-b3d9-08e4e19475db None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:58.058167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b639-ac77-494a-b3d9-08e4e19475db None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:58.058897 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5b639-ac77-494a-b3d9-08e4e19475db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:58.059201 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5b639-ac77-494a-b3d9-08e4e19475db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:58.062777 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5b639-ac77-494a-b3d9-08e4e19475db None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:58.066392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3cdb56-f318-4225-999f-122a122cb5d4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:58.089676 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 366/765] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:58.098939 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5b639-ac77-494a-b3d9-08e4e19475db None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:58.123114 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489212a-9077-47af-9e77-74753c488434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:58.124061 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489212a-9077-47af-9e77-74753c488434 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:58.124344 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489212a-9077-47af-9e77-74753c488434 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:58.124668 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489212a-9077-47af-9e77-74753c488434 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:58.125423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489212a-9077-47af-9e77-74753c488434 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:58.126868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489212a-9077-47af-9e77-74753c488434 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:58.139658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4489212a-9077-47af-9e77-74753c488434 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 21:03:58.139658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:58.139658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:58.139658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:58.139658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:58.139658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:58.139658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:58.139658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:58.139658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 21:03:58.139658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:58.139658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:58.139658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:58.139658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:58.139658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:58.139658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:58.139658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:58.140603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:58.140603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:58.140603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:58.140603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:58.140603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:58.140603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:58.140603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:58.140603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:58.140603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:58.140603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:58.140603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:58.140603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:58.140603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:58.140603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:58.140603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:58.140603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:58.141542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:58.141542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:58.141542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:58.141542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:58.141542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:58.141542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:58.141542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:58.141542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:58.141542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:58.141542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:58.141542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:58.141542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:58.141542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:58.141542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 21:03:58.141542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:58.141542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 400/766] 10.222.0.27 () {56 vars in 1129 bytes} [Mon Jun 27 21:03:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:58.146866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489212a-9077-47af-9e77-74753c488434 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:58.170925 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:58.271080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5b639-ac77-494a-b3d9-08e4e19475db None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:58.312659 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e717f0-76a7-4acd-b49b-d15fb6bcc93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:58.313831 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e717f0-76a7-4acd-b49b-d15fb6bcc93d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:58.314263 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e717f0-76a7-4acd-b49b-d15fb6bcc93d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:58.314685 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e717f0-76a7-4acd-b49b-d15fb6bcc93d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:58.328509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e717f0-76a7-4acd-b49b-d15fb6bcc93d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:58.330575 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e717f0-76a7-4acd-b49b-d15fb6bcc93d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:58.358727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-31e717f0-76a7-4acd-b49b-d15fb6bcc93d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:58.358727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:58.358727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:58.358727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:58.358727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:58.358727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:58.358727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:58.358727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:58.358727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:58.358727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:58.358727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:58.358727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:58.358727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:58.358727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:58.358727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:58.359853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:58.359853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:58.359853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:58.359853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:58.359853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:58.359853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:58.359853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:58.359853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:58.359853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:58.359853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:58.359853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:58.359853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:58.359853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:58.359853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:58.359853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:58.359853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:58.359853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:58.361299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:58.361299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:58.361299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:58.361299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:58.361299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:58.361299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:58.361299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:58.361299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:03:58.366418 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 367/767] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:58.372051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e717f0-76a7-4acd-b49b-d15fb6bcc93d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:58.375107 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:58.383123 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489212a-9077-47af-9e77-74753c488434 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:58.453489 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2866ee5-d841-4ca8-a2d5-7cda1a8fd888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:58.454183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2866ee5-d841-4ca8-a2d5-7cda1a8fd888 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:58.454549 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2866ee5-d841-4ca8-a2d5-7cda1a8fd888 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:58.454826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2866ee5-d841-4ca8-a2d5-7cda1a8fd888 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:58.455536 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2866ee5-d841-4ca8-a2d5-7cda1a8fd888 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:58.455893 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2866ee5-d841-4ca8-a2d5-7cda1a8fd888 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:58.457287 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2866ee5-d841-4ca8-a2d5-7cda1a8fd888 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:58.467536 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 401/768] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Jun 27 21:03:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:58.472298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2866ee5-d841-4ca8-a2d5-7cda1a8fd888 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:58.521375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e717f0-76a7-4acd-b49b-d15fb6bcc93d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:58.558712 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c473562-931b-4b6c-8f89-3683dac36130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:58.559385 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c473562-931b-4b6c-8f89-3683dac36130 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:58.559708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c473562-931b-4b6c-8f89-3683dac36130 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:58.559983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c473562-931b-4b6c-8f89-3683dac36130 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:58.560768 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c473562-931b-4b6c-8f89-3683dac36130 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:58.561077 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c473562-931b-4b6c-8f89-3683dac36130 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:58.562523 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c473562-931b-4b6c-8f89-3683dac36130 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:58.577505 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 368/769] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:58.583789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c473562-931b-4b6c-8f89-3683dac36130 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:58.627469 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:58.653855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:58.660015 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2866ee5-d841-4ca8-a2d5-7cda1a8fd888 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:58.695913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621df29e-af2e-41d3-904d-d1740dcca9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:58.696841 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621df29e-af2e-41d3-904d-d1740dcca9dc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:58.697129 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621df29e-af2e-41d3-904d-d1740dcca9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:58.697409 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621df29e-af2e-41d3-904d-d1740dcca9dc None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:58.698211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621df29e-af2e-41d3-904d-d1740dcca9dc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:58.698522 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621df29e-af2e-41d3-904d-d1740dcca9dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:58.699168 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c473562-931b-4b6c-8f89-3683dac36130 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:58.700057 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621df29e-af2e-41d3-904d-d1740dcca9dc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:58.705732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 402/770] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Jun 27 21:03:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:58.707063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e86ea8-6126-4011-93c7-cf0b973c150e None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:58.707351 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e86ea8-6126-4011-93c7-cf0b973c150e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:58.707638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e86ea8-6126-4011-93c7-cf0b973c150e None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:58.709161 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 403/771] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:58.714032 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a731d-9ee3-40e0-aa85-c197bb681f5e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:58.714317 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a731d-9ee3-40e0-aa85-c197bb681f5e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:58.714596 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a731d-9ee3-40e0-aa85-c197bb681f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:58.733276 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a11a731d-9ee3-40e0-aa85-c197bb681f5e None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:58.734481 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a11a731d-9ee3-40e0-aa85-c197bb681f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:58.749981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33fee44-d6bd-423d-9442-1ddfa797d5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:58.750665 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fee44-d6bd-423d-9442-1ddfa797d5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:58.750950 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fee44-d6bd-423d-9442-1ddfa797d5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:58.751573 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fee44-d6bd-423d-9442-1ddfa797d5a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:58.752929 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33fee44-d6bd-423d-9442-1ddfa797d5a9 None admin] RBAC: Adding query filter params (role.id=5c3c095bb53943eca95c4e049cd8cbf5, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=f28acb0322114aebafcf62bdf9a346df) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:58.753250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33fee44-d6bd-423d-9442-1ddfa797d5a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:58.755243 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33fee44-d6bd-423d-9442-1ddfa797d5a9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:58.762940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 369/772] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:58 2022] GET /identity/v3/role_assignments?user.id=f28acb0322114aebafcf62bdf9a346df&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=5c3c095bb53943eca95c4e049cd8cbf5 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:58.764360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33fee44-d6bd-423d-9442-1ddfa797d5a9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:58.870038 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 404/773] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:03:58.907365 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a731d-9ee3-40e0-aa85-c197bb681f5e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:58.928323 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33fee44-d6bd-423d-9442-1ddfa797d5a9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:58.967716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:58.971981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd1421-250f-4c5f-8736-345e7cda2f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:58.972672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd1421-250f-4c5f-8736-345e7cda2f5d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:58.972966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd1421-250f-4c5f-8736-345e7cda2f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:58.973251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd1421-250f-4c5f-8736-345e7cda2f5d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:58.974119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd1421-250f-4c5f-8736-345e7cda2f5d None admin] RBAC: Adding query filter params (interface=public, service_id=f77b769c79204af99f9319269d185805, region_id=RegionOne) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:58.977822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd1421-250f-4c5f-8736-345e7cda2f5d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:58.979451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd1421-250f-4c5f-8736-345e7cda2f5d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:58.996297 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 370/774] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Jun 27 21:03:58 2022] GET /identity/v3/endpoints?service_id=f77b769c79204af99f9319269d185805&interface=public®ion_id=RegionOne => generated 194 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:59.001968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd1421-250f-4c5f-8736-345e7cda2f5d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:59.037371 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:59.110760 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a731d-9ee3-40e0-aa85-c197bb681f5e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:59.117869 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd1421-250f-4c5f-8736-345e7cda2f5d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:59.152575 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b83bf3-17f7-4d51-a2f8-93da4ed321b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:59.153355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b83bf3-17f7-4d51-a2f8-93da4ed321b9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:59.153714 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b83bf3-17f7-4d51-a2f8-93da4ed321b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:59.154066 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b83bf3-17f7-4d51-a2f8-93da4ed321b9 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:59.154888 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b83bf3-17f7-4d51-a2f8-93da4ed321b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:59.156413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b83bf3-17f7-4d51-a2f8-93da4ed321b9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:59.162600 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 371/775] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Jun 27 21:03:59 2022] GET /identity/v3/services => generated 772 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:59.171218 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee9e1b-603e-42bb-9d2a-f88da902f318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:59.172170 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee9e1b-603e-42bb-9d2a-f88da902f318 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:59.172450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee9e1b-603e-42bb-9d2a-f88da902f318 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:59.172720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee9e1b-603e-42bb-9d2a-f88da902f318 None admin] PATH_INFO: `/v3/services/network` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:59.173443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee9e1b-603e-42bb-9d2a-f88da902f318 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:59.174848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee9e1b-603e-42bb-9d2a-f88da902f318 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:59.187901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4fee9e1b-603e-42bb-9d2a-f88da902f318 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 21:03:59.187901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:59.187901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:59.187901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:59.187901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:59.187901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:59.187901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:59.187901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:59.187901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 21:03:59.187901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:59.187901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:59.187901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:59.187901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:59.187901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:59.187901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:59.187901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:59.187901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:59.189043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:59.189043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:59.189043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:59.189043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:59.189043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:59.189043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:59.189043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:59.189043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:59.189043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:59.189043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:59.189043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:59.189043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:59.189043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:59.189043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:59.189043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:59.189043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:59.189858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:59.189858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:59.189858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:59.189858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:59.189858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:59.189858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:59.189858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:59.189858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:59.189858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:59.189858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:59.189858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:59.189858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:59.189858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 21:03:59.189858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:03:59.190570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 405/776] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:58 2022] GET /identity/v3/services/network => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:59.199533 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b83bf3-17f7-4d51-a2f8-93da4ed321b9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:59.302917 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:59.356045 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b83bf3-17f7-4d51-a2f8-93da4ed321b9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:59.394266 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4a0c2-54a5-4249-bf83-aac10dc347e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:59.395013 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4a0c2-54a5-4249-bf83-aac10dc347e7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:59.395225 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4a0c2-54a5-4249-bf83-aac10dc347e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:59.395503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4a0c2-54a5-4249-bf83-aac10dc347e7 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:59.396254 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4a0c2-54a5-4249-bf83-aac10dc347e7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:59.396568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4a0c2-54a5-4249-bf83-aac10dc347e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:59.398019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4a0c2-54a5-4249-bf83-aac10dc347e7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:59.403423 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 372/777] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Jun 27 21:03:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:59.409317 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee9e1b-603e-42bb-9d2a-f88da902f318 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:59.533650 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee9e1b-603e-42bb-9d2a-f88da902f318 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:59.573942 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:59.582197 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a08cf8-675c-43f8-bf4d-079273736d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:59.582878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a08cf8-675c-43f8-bf4d-079273736d5f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:59.583161 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a08cf8-675c-43f8-bf4d-079273736d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:59.583438 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a08cf8-675c-43f8-bf4d-079273736d5f None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:59.584184 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a08cf8-675c-43f8-bf4d-079273736d5f None admin] RBAC: Adding query filter params (type=network) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:59.584495 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a08cf8-675c-43f8-bf4d-079273736d5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:59.585938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a08cf8-675c-43f8-bf4d-079273736d5f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:59.591259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 406/778] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Jun 27 21:03:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:59.596968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4a0c2-54a5-4249-bf83-aac10dc347e7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:59.704752 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4a0c2-54a5-4249-bf83-aac10dc347e7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:59.723271 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:59.741954 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d27e0e-1b7c-4c16-8da4-1698c1fe02db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:59.742627 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d27e0e-1b7c-4c16-8da4-1698c1fe02db None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:59.742911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d27e0e-1b7c-4c16-8da4-1698c1fe02db None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:59.743190 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d27e0e-1b7c-4c16-8da4-1698c1fe02db None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:59.744021 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d27e0e-1b7c-4c16-8da4-1698c1fe02db None admin] RBAC: Adding query filter params (interface=public, service_id=b9b389a2f0b340e18ad02c08c03c597d, region_id=RegionOne) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:59.744336 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d27e0e-1b7c-4c16-8da4-1698c1fe02db None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:59.745727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d27e0e-1b7c-4c16-8da4-1698c1fe02db None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:59.751709 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 373/779] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Jun 27 21:03:59 2022] GET /identity/v3/endpoints?service_id=b9b389a2f0b340e18ad02c08c03c597d&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:59.757134 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a08cf8-675c-43f8-bf4d-079273736d5f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:59.855715 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b6c4c-7566-42fa-9b2b-594ee0047010 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:59.856336 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b6c4c-7566-42fa-9b2b-594ee0047010 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:59.856818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b6c4c-7566-42fa-9b2b-594ee0047010 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:59.858634 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 374/780] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:03:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:59.863856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a291fed-716b-40a7-9786-b369d8ab35d6 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:59.864425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a291fed-716b-40a7-9786-b369d8ab35d6 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:59.864905 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a291fed-716b-40a7-9786-b369d8ab35d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:59.884043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a08cf8-675c-43f8-bf4d-079273736d5f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:59.900887 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-7a291fed-716b-40a7-9786-b369d8ab35d6 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:59.902117 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7a291fed-716b-40a7-9786-b369d8ab35d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:59.951209 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc380f5-6a8d-4181-b650-bc2243d19353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:59.951924 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc380f5-6a8d-4181-b650-bc2243d19353 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:59.952208 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc380f5-6a8d-4181-b650-bc2243d19353 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:59.952490 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc380f5-6a8d-4181-b650-bc2243d19353 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:59.953185 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc380f5-6a8d-4181-b650-bc2243d19353 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:59.954651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc380f5-6a8d-4181-b650-bc2243d19353 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:59.968586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 407/781] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Jun 27 21:03:59 2022] GET /identity/v3/services => generated 772 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:59.993772 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 375/782] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:00.033088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc380f5-6a8d-4181-b650-bc2243d19353 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:00.108162 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:00.172582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc380f5-6a8d-4181-b650-bc2243d19353 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:00.183642 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0c2ba-879a-45c3-9cfd-1bd538887c24 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:00.183642 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0c2ba-879a-45c3-9cfd-1bd538887c24 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:00.183642 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0c2ba-879a-45c3-9cfd-1bd538887c24 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:00.184839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 376/783] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:00.190799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26474679-434c-4e14-b56f-3f28e7064041 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:00.191089 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26474679-434c-4e14-b56f-3f28e7064041 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:00.191465 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26474679-434c-4e14-b56f-3f28e7064041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:00.210152 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f735175-c7b5-428b-912b-7b77d2be419f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:00.210966 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f735175-c7b5-428b-912b-7b77d2be419f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:00.211248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f735175-c7b5-428b-912b-7b77d2be419f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:00.211520 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f735175-c7b5-428b-912b-7b77d2be419f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:00.212417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-26474679-434c-4e14-b56f-3f28e7064041 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:00.213755 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-26474679-434c-4e14-b56f-3f28e7064041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:00.223860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f735175-c7b5-428b-912b-7b77d2be419f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:00.233539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-1f735175-c7b5-428b-912b-7b77d2be419f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:04:00.233539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:00.233539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:00.233539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:00.233539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:00.233539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:00.233539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:00.233539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:00.233539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:00.233539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:00.233539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:00.233539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:00.233539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:00.233539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:00.233539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:00.233539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:00.234450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:00.234450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:00.234450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:00.234450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:00.234450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:00.234450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:00.234450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:00.234450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:00.234450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:00.234450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:00.234450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:00.234450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:00.234450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:00.234450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:00.234450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:00.234450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:00.235860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:00.235860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:00.235860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:00.235860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:00.235860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:00.235860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:00.235860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:00.235860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:00.235860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:00.235860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:00.235860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:04:00.235860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:00.235860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 408/784] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:04:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:00.244603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f735175-c7b5-428b-912b-7b77d2be419f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:00.313523 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 377/785] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:00.377001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26474679-434c-4e14-b56f-3f28e7064041 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:00.403867 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f735175-c7b5-428b-912b-7b77d2be419f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:00.425622 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:00.450220 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1c1ae-4026-48ef-9943-9e136c4f8e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:00.450908 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1c1ae-4026-48ef-9943-9e136c4f8e11 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:00.451195 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1c1ae-4026-48ef-9943-9e136c4f8e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:00.451471 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1c1ae-4026-48ef-9943-9e136c4f8e11 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:00.452258 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1c1ae-4026-48ef-9943-9e136c4f8e11 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:00.452708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1c1ae-4026-48ef-9943-9e136c4f8e11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:00.454282 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1c1ae-4026-48ef-9943-9e136c4f8e11 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:00.463225 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 409/786] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:04:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:00.470001 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1c1ae-4026-48ef-9943-9e136c4f8e11 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:00.492044 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:00.580167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26474679-434c-4e14-b56f-3f28e7064041 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:00.626692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255776e-37c9-40c6-8d21-9e2ae1fe30ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:00.627576 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255776e-37c9-40c6-8d21-9e2ae1fe30ee None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:00.627828 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255776e-37c9-40c6-8d21-9e2ae1fe30ee None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:00.628108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255776e-37c9-40c6-8d21-9e2ae1fe30ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:00.638462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1c1ae-4026-48ef-9943-9e136c4f8e11 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:00.645225 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255776e-37c9-40c6-8d21-9e2ae1fe30ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:00.661322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-6255776e-37c9-40c6-8d21-9e2ae1fe30ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:00.661322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:00.661322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:00.661322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:00.661322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:00.661322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:00.661322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:00.661322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:00.661322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:00.661322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:00.661322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:00.661322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:00.661322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:00.661322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:00.661322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:00.661322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:00.662270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:00.662270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:00.662270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:00.662270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:00.662270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:00.662270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:00.662270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:00.662270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:00.662270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:00.662270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:00.662270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:00.662270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:00.662270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:00.662270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:00.662270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:00.662270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:00.663092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:00.663092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:00.663092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:00.663092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:00.663092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:00.663092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:00.663092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:00.663092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:00.663092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:00.663092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:00.663092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:00.663092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:00.663092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 378/787] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:04:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:00.668621 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255776e-37c9-40c6-8d21-9e2ae1fe30ee None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:00.684630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f0ce8-83e6-43fb-8716-de99340fcc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:00.685866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f0ce8-83e6-43fb-8716-de99340fcc7b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:00.686353 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f0ce8-83e6-43fb-8716-de99340fcc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:00.686828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f0ce8-83e6-43fb-8716-de99340fcc7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:00.703961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f0ce8-83e6-43fb-8716-de99340fcc7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:00.705949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f0ce8-83e6-43fb-8716-de99340fcc7b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:00.728883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-a29f0ce8-83e6-43fb-8716-de99340fcc7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:00.728883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:00.728883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:00.728883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:00.728883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:00.728883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:00.728883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:00.728883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:00.728883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:00.728883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:00.728883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:00.728883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:00.728883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:00.728883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:00.728883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:00.729838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:00.729838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:00.729838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:00.729838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:00.729838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:00.729838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:00.729838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:00.729838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:00.729838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:00.729838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:00.729838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:00.729838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:00.729838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:00.729838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:00.729838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:00.729838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:00.729838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:00.730736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:00.730736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:00.730736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:00.730736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:00.735161 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 410/788] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:00.757729 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f0ce8-83e6-43fb-8716-de99340fcc7b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:00.821564 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255776e-37c9-40c6-8d21-9e2ae1fe30ee None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:00.824624 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:00.831691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:00.857595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920f1ed-08fc-4dc7-8881-6af9da8862c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:00.858314 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920f1ed-08fc-4dc7-8881-6af9da8862c9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:00.858599 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920f1ed-08fc-4dc7-8881-6af9da8862c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:00.858876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920f1ed-08fc-4dc7-8881-6af9da8862c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:00.859664 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920f1ed-08fc-4dc7-8881-6af9da8862c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:00.859981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920f1ed-08fc-4dc7-8881-6af9da8862c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:00.861504 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920f1ed-08fc-4dc7-8881-6af9da8862c9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:00.870190 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 379/789] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:04:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:00.875688 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920f1ed-08fc-4dc7-8881-6af9da8862c9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:00.969637 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f0ce8-83e6-43fb-8716-de99340fcc7b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:00.978179 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920f1ed-08fc-4dc7-8881-6af9da8862c9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:01.014785 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f36e6e-e903-4034-aa9c-f83d5258e125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:01.015616 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f36e6e-e903-4034-aa9c-f83d5258e125 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:01.016050 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f36e6e-e903-4034-aa9c-f83d5258e125 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:01.016333 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f36e6e-e903-4034-aa9c-f83d5258e125 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:01.028748 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f36e6e-e903-4034-aa9c-f83d5258e125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:01.030234 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f36e6e-e903-4034-aa9c-f83d5258e125 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:01.051226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-a2f36e6e-e903-4034-aa9c-f83d5258e125 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:01.051226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:01.051226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:01.051226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:01.051226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:01.051226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:01.051226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:01.051226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:01.051226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:01.051226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:01.051226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:01.051226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:01.051226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:01.051226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:01.051226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:01.052371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:01.052371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:01.052371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:01.052371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:01.052371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:01.052371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:01.052371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:01.052371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:01.052371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:01.052371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:01.052371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:01.052371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:01.052371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:01.052371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:01.052371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:01.052371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:01.052371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:01.053350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:01.053350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:01.053350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:01.053350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:01.054158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 380/790] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:01.060684 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f36e6e-e903-4034-aa9c-f83d5258e125 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:01.088062 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:01.104926 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352da1f-0e21-46e1-a76f-b1686c9c50da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:01.105622 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352da1f-0e21-46e1-a76f-b1686c9c50da None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:01.105900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352da1f-0e21-46e1-a76f-b1686c9c50da None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:01.106170 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352da1f-0e21-46e1-a76f-b1686c9c50da None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:01.106899 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352da1f-0e21-46e1-a76f-b1686c9c50da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:01.107203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352da1f-0e21-46e1-a76f-b1686c9c50da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:01.108621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352da1f-0e21-46e1-a76f-b1686c9c50da None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:01.141898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 411/791] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:01.152572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352da1f-0e21-46e1-a76f-b1686c9c50da None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:01.180922 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f36e6e-e903-4034-aa9c-f83d5258e125 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:01.219559 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb17f0a-0c14-4972-9126-b4d7effabade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:01.220425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb17f0a-0c14-4972-9126-b4d7effabade None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:01.220656 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb17f0a-0c14-4972-9126-b4d7effabade None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:01.220940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb17f0a-0c14-4972-9126-b4d7effabade None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:01.221676 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb17f0a-0c14-4972-9126-b4d7effabade None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:01.222128 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb17f0a-0c14-4972-9126-b4d7effabade None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:01.223535 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb17f0a-0c14-4972-9126-b4d7effabade None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:01.237177 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 381/792] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:01.242756 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb17f0a-0c14-4972-9126-b4d7effabade None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:01.348015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:01.352139 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb17f0a-0c14-4972-9126-b4d7effabade None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:01.405192 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7e992-de2c-41ba-af91-72be564c1e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:01.406031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7e992-de2c-41ba-af91-72be564c1e21 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:01.406315 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7e992-de2c-41ba-af91-72be564c1e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:01.406594 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7e992-de2c-41ba-af91-72be564c1e21 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:01.416455 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee7e992-de2c-41ba-af91-72be564c1e21 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:01.417959 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee7e992-de2c-41ba-af91-72be564c1e21 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:01.426677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:01.433879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-cee7e992-de2c-41ba-af91-72be564c1e21 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:04:01.433879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:01.433879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:01.433879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:01.433879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:01.433879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:01.433879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:01.433879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:01.433879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:01.433879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:01.433879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:01.433879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:01.433879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:04:01.433879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:04:01.433879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:04:01.433879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:04:01.434820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:01.434820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:01.434820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:04:01.434820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:01.434820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:04:01.434820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:01.434820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:01.434820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:01.434820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:01.434820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:01.434820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:01.434820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:01.434820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:01.434820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:01.434820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:01.434820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:01.434820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:01.434820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:01.436216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:01.436216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:01.436216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:04:01.436216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:04:01.436216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:04:01.436216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:04:01.436216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:04:01.436216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:04:01.436216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:04:01.436216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:01.436216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 382/793] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Jun 27 21:04:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:01.436813 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf6d86-55ad-4134-be7d-0a3b87afc026 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:01.437109 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf6d86-55ad-4134-be7d-0a3b87afc026 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:01.437323 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf6d86-55ad-4134-be7d-0a3b87afc026 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:01.442677 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 383/794] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:01.443108 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352da1f-0e21-46e1-a76f-b1686c9c50da None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:01.443430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf6d86-55ad-4134-be7d-0a3b87afc026 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:01.488388 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3ef10-4894-4383-96de-0b1d99b848d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:01.489146 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3ef10-4894-4383-96de-0b1d99b848d7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:01.489746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3ef10-4894-4383-96de-0b1d99b848d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:01.490034 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3ef10-4894-4383-96de-0b1d99b848d7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:01.499657 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3ef10-4894-4383-96de-0b1d99b848d7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:01.501490 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3ef10-4894-4383-96de-0b1d99b848d7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:01.517360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-aec3ef10-4894-4383-96de-0b1d99b848d7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:04:01.517360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:01.517360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:01.517360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:01.517360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:01.517360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:01.517360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:01.517360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:01.517360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:01.517360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:01.517360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:01.517360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:01.517360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:04:01.517360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:04:01.517360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:04:01.517360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:04:01.519581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:01.519581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:01.519581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:04:01.519581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:01.519581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:04:01.519581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:01.519581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:01.519581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:01.519581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:01.519581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:01.519581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:01.519581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:01.519581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:01.519581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:01.519581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:01.519581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:01.519581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:01.519581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:01.521927 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:01.521927 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:01.521927 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:04:01.521927 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:04:01.521927 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:04:01.521927 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:04:01.521927 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:04:01.521927 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:04:01.521927 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:04:01.521927 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:01.521927 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 412/795] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Jun 27 21:04:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:01.527627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753c07d-5649-4958-8900-3fe5385b4d2c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:01.527627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753c07d-5649-4958-8900-3fe5385b4d2c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:01.527627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753c07d-5649-4958-8900-3fe5385b4d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:01.542229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0753c07d-5649-4958-8900-3fe5385b4d2c None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:01.543427 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0753c07d-5649-4958-8900-3fe5385b4d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:01.548979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf6d86-55ad-4134-be7d-0a3b87afc026 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:01.584327 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7b3b3-7b6a-4494-8338-84643cac508f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:01.585001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7b3b3-7b6a-4494-8338-84643cac508f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:01.585285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7b3b3-7b6a-4494-8338-84643cac508f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:01.585553 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7b3b3-7b6a-4494-8338-84643cac508f None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:01.586389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7b3b3-7b6a-4494-8338-84643cac508f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:01.586694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7b3b3-7b6a-4494-8338-84643cac508f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:01.588165 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7b3b3-7b6a-4494-8338-84643cac508f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:01.591045 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:01.608301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 384/796] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Jun 27 21:04:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:01.609140 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7b3b3-7b6a-4494-8338-84643cac508f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:01.620207 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 413/797] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:01.621074 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753c07d-5649-4958-8900-3fe5385b4d2c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:01.706874 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:01.728210 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753c07d-5649-4958-8900-3fe5385b4d2c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:01.741312 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7b3b3-7b6a-4494-8338-84643cac508f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:01.765035 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad0c49-1cc4-42fc-8857-0293cc199fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:01.765697 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad0c49-1cc4-42fc-8857-0293cc199fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:01.765981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad0c49-1cc4-42fc-8857-0293cc199fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:01.766257 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad0c49-1cc4-42fc-8857-0293cc199fe1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:01.778181 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ff08b-a390-43a5-bc9d-e3a840c8170a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:01.778861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ff08b-a390-43a5-bc9d-e3a840c8170a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:01.779146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ff08b-a390-43a5-bc9d-e3a840c8170a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:01.779420 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ff08b-a390-43a5-bc9d-e3a840c8170a None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:01.780300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ff08b-a390-43a5-bc9d-e3a840c8170a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:01.780613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ff08b-a390-43a5-bc9d-e3a840c8170a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:01.782073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ff08b-a390-43a5-bc9d-e3a840c8170a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:01.801500 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 385/798] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Jun 27 21:04:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:01.804480 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ff08b-a390-43a5-bc9d-e3a840c8170a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:01.838615 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faad0c49-1cc4-42fc-8857-0293cc199fe1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:01.840157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faad0c49-1cc4-42fc-8857-0293cc199fe1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:01.863373 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:01.915703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ff08b-a390-43a5-bc9d-e3a840c8170a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:01.924924 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:01.946081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 414/799] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:01 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:01.947428 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c517ca9-cf33-4674-b046-ca0cbde2d126 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:01.947751 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c517ca9-cf33-4674-b046-ca0cbde2d126 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:01.948031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c517ca9-cf33-4674-b046-ca0cbde2d126 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:01.949297 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 415/800] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:01.949968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c517ca9-cf33-4674-b046-ca0cbde2d126 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:01.963777 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bcf7c8-cf58-4c02-bc1a-f1a5eff695a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:01.964601 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bcf7c8-cf58-4c02-bc1a-f1a5eff695a8 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:01.965203 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bcf7c8-cf58-4c02-bc1a-f1a5eff695a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:01.965485 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bcf7c8-cf58-4c02-bc1a-f1a5eff695a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:01.980648 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bcf7c8-cf58-4c02-bc1a-f1a5eff695a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:01.988453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-99bcf7c8-cf58-4c02-bc1a-f1a5eff695a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:01.988453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:01.988453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:01.988453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:01.988453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:01.988453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:01.988453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:01.988453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:01.988453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:01.988453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:01.988453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:01.988453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:01.988453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:01.988453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:01.988453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:01.988453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:01.989452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:01.989452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:01.989452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:01.989452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:01.989452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:01.989452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:01.989452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:01.989452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:01.989452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:01.989452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:01.989452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:01.989452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:01.989452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:01.989452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:01.989452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:01.989452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:01.990277 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:01.990277 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:01.990277 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:01.990277 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:01.990277 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:01.990277 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:01.990277 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:01.990277 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:01.990277 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:01.990277 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:01.990277 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:01.990277 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:01.994872 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 386/801] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:04:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:01.995752 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bcf7c8-cf58-4c02-bc1a-f1a5eff695a8 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:02.070005 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c517ca9-cf33-4674-b046-ca0cbde2d126 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:02.100728 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bcf7c8-cf58-4c02-bc1a-f1a5eff695a8 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:02.106686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d462a3-bc50-4b36-98a7-1a2a78d4041f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:02.107490 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d462a3-bc50-4b36-98a7-1a2a78d4041f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:02.107800 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d462a3-bc50-4b36-98a7-1a2a78d4041f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:02.108073 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d462a3-bc50-4b36-98a7-1a2a78d4041f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:02.117023 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:02.118782 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d462a3-bc50-4b36-98a7-1a2a78d4041f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:02.120380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d462a3-bc50-4b36-98a7-1a2a78d4041f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:02.137898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-b5d462a3-bc50-4b36-98a7-1a2a78d4041f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:02.137898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:02.137898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:02.137898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:02.137898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:02.137898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:02.137898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:02.137898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:02.137898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:02.137898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:02.137898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:02.137898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:02.137898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:02.137898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:02.137898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:02.138822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:02.138822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:02.138822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:02.138822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:02.138822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:02.138822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:02.138822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:02.138822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:02.138822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:02.138822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:02.138822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:02.138822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:02.138822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:02.138822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:02.138822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:02.138822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:02.138822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:02.144690 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:02.144690 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:02.144690 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:02.144690 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:02.144690 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 416/802] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:02.144690 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff1ca2-7c7d-41a0-8032-52df13402202 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:02.144690 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff1ca2-7c7d-41a0-8032-52df13402202 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:02.144690 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff1ca2-7c7d-41a0-8032-52df13402202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:02.145129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ac20f-91a3-469b-9c01-a52e1e2392e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:02.145129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ac20f-91a3-469b-9c01-a52e1e2392e2 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:02.145129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ac20f-91a3-469b-9c01-a52e1e2392e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:02.145129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ac20f-91a3-469b-9c01-a52e1e2392e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:02.156156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731ac20f-91a3-469b-9c01-a52e1e2392e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:02.157325 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b8ff1ca2-7c7d-41a0-8032-52df13402202 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:02.158201 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731ac20f-91a3-469b-9c01-a52e1e2392e2 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:02.158823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b8ff1ca2-7c7d-41a0-8032-52df13402202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:02.177184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-731ac20f-91a3-469b-9c01-a52e1e2392e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:02.177184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:02.177184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:02.177184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:02.177184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:02.177184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:02.177184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:02.177184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:02.177184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:02.177184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:02.177184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:02.177184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:02.177184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:02.177184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:02.177184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:02.178076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:02.178076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:02.178076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:02.178076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:02.178076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:02.178076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:02.178076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:02.178076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:02.178076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:02.178076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:02.178076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:02.178076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:02.178076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:02.178076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:02.178076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:02.178076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:02.178076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:02.178949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:02.178949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:02.178949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:02.178949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:02.178949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 387/803] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:02.179703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ac20f-91a3-469b-9c01-a52e1e2392e2 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:02.188475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:02.234081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 417/804] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:02.235201 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff1ca2-7c7d-41a0-8032-52df13402202 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:02.300142 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ac20f-91a3-469b-9c01-a52e1e2392e2 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:02.342824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8870cde-03b4-47f2-afeb-333a2e95c5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:02.343600 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8870cde-03b4-47f2-afeb-333a2e95c5ab None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:02.343908 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8870cde-03b4-47f2-afeb-333a2e95c5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:02.344177 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8870cde-03b4-47f2-afeb-333a2e95c5ab None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:02.344936 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8870cde-03b4-47f2-afeb-333a2e95c5ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:02.345243 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8870cde-03b4-47f2-afeb-333a2e95c5ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:02.346628 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8870cde-03b4-47f2-afeb-333a2e95c5ab None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:02.348064 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff1ca2-7c7d-41a0-8032-52df13402202 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:02.354962 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 388/805] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:04:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:02.355765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8870cde-03b4-47f2-afeb-333a2e95c5ab None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:02.382379 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:02.383243 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df941210-e79b-4755-897c-d9291424d7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:02.383942 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df941210-e79b-4755-897c-d9291424d7ea None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:02.384227 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df941210-e79b-4755-897c-d9291424d7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:02.384554 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df941210-e79b-4755-897c-d9291424d7ea None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:02.385397 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df941210-e79b-4755-897c-d9291424d7ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:02.385708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df941210-e79b-4755-897c-d9291424d7ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:02.387109 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df941210-e79b-4755-897c-d9291424d7ea None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:02.401923 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 418/806] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:02.402846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df941210-e79b-4755-897c-d9291424d7ea None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:02.440943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:02.475791 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8870cde-03b4-47f2-afeb-333a2e95c5ab None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:02.508688 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86853e1d-f405-470a-9f6c-e88193dbfee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:02.509349 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86853e1d-f405-470a-9f6c-e88193dbfee2 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:02.509638 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86853e1d-f405-470a-9f6c-e88193dbfee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:02.509900 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86853e1d-f405-470a-9f6c-e88193dbfee2 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:02.510620 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86853e1d-f405-470a-9f6c-e88193dbfee2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:02.510910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86853e1d-f405-470a-9f6c-e88193dbfee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:02.512336 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86853e1d-f405-470a-9f6c-e88193dbfee2 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:02.519621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df941210-e79b-4755-897c-d9291424d7ea None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:02.523780 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 389/807] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:02.525070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3464faf-a3dc-46c2-8f26-0374671a450a None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:02.525370 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3464faf-a3dc-46c2-8f26-0374671a450a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:02.525639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3464faf-a3dc-46c2-8f26-0374671a450a None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:02.526843 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 390/808] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:02.531020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3464faf-a3dc-46c2-8f26-0374671a450a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:02.557463 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188698dd-a54c-4657-8190-c612ec3d083f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:02.558288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188698dd-a54c-4657-8190-c612ec3d083f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:02.558575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188698dd-a54c-4657-8190-c612ec3d083f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:02.558853 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188698dd-a54c-4657-8190-c612ec3d083f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:02.559503 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188698dd-a54c-4657-8190-c612ec3d083f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:02.560911 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188698dd-a54c-4657-8190-c612ec3d083f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:02.573081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-188698dd-a54c-4657-8190-c612ec3d083f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 21:04:02.573081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:02.573081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:02.573081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:02.573081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:02.573081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:02.573081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:02.573081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:02.573081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 21:04:02.573081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:02.573081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:02.573081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:02.573081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:02.573081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:02.573081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:02.573081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:02.573955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:02.573955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:02.573955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:02.573955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:02.573955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:02.573955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:02.573955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:02.573955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:02.573955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:02.573955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:02.573955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:02.573955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:02.573955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:02.573955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:02.573955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:02.573955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:02.574753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:02.574753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:02.574753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:02.574753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:02.574753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:02.574753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:02.574753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:02.574753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:02.574753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:02.574753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:02.574753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:02.574753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:02.574753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:02.574753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 21:04:02.574753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:02.574753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 419/809] 10.222.0.27 () {56 vars in 1129 bytes} [Mon Jun 27 21:04:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:02.575589 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188698dd-a54c-4657-8190-c612ec3d083f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:02.619070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:02.634931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3464faf-a3dc-46c2-8f26-0374671a450a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:02.664478 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:02.669970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7907b-60ae-49a7-984d-7de6b917617b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:02.670766 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7907b-60ae-49a7-984d-7de6b917617b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:02.671034 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7907b-60ae-49a7-984d-7de6b917617b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:02.671303 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7907b-60ae-49a7-984d-7de6b917617b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:02.679552 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188698dd-a54c-4657-8190-c612ec3d083f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:02.681397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7907b-60ae-49a7-984d-7de6b917617b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:02.682875 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7907b-60ae-49a7-984d-7de6b917617b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:02.700916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-17c7907b-60ae-49a7-984d-7de6b917617b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:02.700916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:02.700916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:02.700916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:02.700916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:02.700916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:02.700916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:02.700916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:02.700916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:02.700916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:02.700916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:02.700916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:02.700916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:02.700916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:02.700916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:02.701802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:02.701802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:02.701802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:02.701802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:02.701802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:02.701802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:02.701802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:02.701802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:02.701802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:02.701802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:02.701802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:02.701802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:02.701802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:02.701802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:02.701802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:02.701802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:02.701802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:02.702669 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:02.702669 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:02.702669 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:02.702669 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:02.702669 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 391/810] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:02.702949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7907b-60ae-49a7-984d-7de6b917617b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:02.719178 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c3bed-1c9a-427f-a489-a1ee0f649692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:02.720053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c3bed-1c9a-427f-a489-a1ee0f649692 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:02.720328 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c3bed-1c9a-427f-a489-a1ee0f649692 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:02.720600 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c3bed-1c9a-427f-a489-a1ee0f649692 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:02.727771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c3bed-1c9a-427f-a489-a1ee0f649692 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:02.729234 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c3bed-1c9a-427f-a489-a1ee0f649692 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:02.743391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f54c3bed-1c9a-427f-a489-a1ee0f649692 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:02.743391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:02.743391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:02.743391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:02.743391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:02.743391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:02.743391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:02.743391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:02.743391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:02.743391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:02.743391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:02.743391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:02.743391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:04:02.743391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:04:02.743391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:04:02.744333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:04:02.744333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:02.744333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:02.744333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:02.744333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:02.744333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:02.744333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:02.744333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:02.744333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:02.744333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:02.744333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:02.744333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:02.744333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:02.744333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:02.744333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:02.744333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:02.744333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:02.745195 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:04:02.745195 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:04:02.745195 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:04:02.745195 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:04:02.745195 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:04:02.745195 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:04:02.745195 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:02.745195 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:02.745195 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 420/811] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:04:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:02.746095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770d7b3-ea21-44ed-bc35-e05383c25a7f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:02.746095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770d7b3-ea21-44ed-bc35-e05383c25a7f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:02.746332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770d7b3-ea21-44ed-bc35-e05383c25a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:02.765164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9770d7b3-ea21-44ed-bc35-e05383c25a7f None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:02.766367 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9770d7b3-ea21-44ed-bc35-e05383c25a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:02.824723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7907b-60ae-49a7-984d-7de6b917617b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:02.839323 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 421/812] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:02.840176 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770d7b3-ea21-44ed-bc35-e05383c25a7f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:02.859976 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d0c42-ef5f-411b-90d2-7f453ee2242e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:02.861101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d0c42-ef5f-411b-90d2-7f453ee2242e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:02.861382 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d0c42-ef5f-411b-90d2-7f453ee2242e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:02.861651 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d0c42-ef5f-411b-90d2-7f453ee2242e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:02.868563 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d0c42-ef5f-411b-90d2-7f453ee2242e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:02.870045 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d0c42-ef5f-411b-90d2-7f453ee2242e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:02.871819 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:02.884892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-633d0c42-ef5f-411b-90d2-7f453ee2242e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:02.884892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:02.884892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:02.884892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:02.884892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:02.884892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:02.884892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:02.884892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:02.884892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:02.884892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:02.884892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:02.884892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:02.884892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:04:02.884892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:04:02.884892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:04:02.885944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:04:02.885944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:02.885944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:02.885944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:02.885944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:02.885944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:02.885944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:02.885944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:02.885944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:02.885944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:02.885944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:02.885944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:02.885944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:02.885944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:02.885944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:02.885944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:02.885944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:02.886934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:04:02.886934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:04:02.886934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:04:02.886934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:04:02.886934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:04:02.886934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:04:02.886934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:02.886934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:02.886934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 392/813] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:04:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:02.886934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d0c42-ef5f-411b-90d2-7f453ee2242e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:02.918022 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:02.941072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770d7b3-ea21-44ed-bc35-e05383c25a7f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:02.975943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ff38e-b4aa-4265-8529-bf9324f60a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:02.976617 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ff38e-b4aa-4265-8529-bf9324f60a1a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:02.976903 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ff38e-b4aa-4265-8529-bf9324f60a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:02.977177 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ff38e-b4aa-4265-8529-bf9324f60a1a None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:02.977880 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ff38e-b4aa-4265-8529-bf9324f60a1a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:02.978189 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ff38e-b4aa-4265-8529-bf9324f60a1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:02.979573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ff38e-b4aa-4265-8529-bf9324f60a1a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:02.984550 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 422/814] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Jun 27 21:04:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:02.985361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ff38e-b4aa-4265-8529-bf9324f60a1a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:02.986911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d0c42-ef5f-411b-90d2-7f453ee2242e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:03.021039 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfc68e-19b0-4580-bbe7-fb12f1edbcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:03.021702 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfc68e-19b0-4580-bbe7-fb12f1edbcce None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:03.021982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfc68e-19b0-4580-bbe7-fb12f1edbcce None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:03.022258 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfc68e-19b0-4580-bbe7-fb12f1edbcce None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:03.022973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dfc68e-19b0-4580-bbe7-fb12f1edbcce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:03.023279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dfc68e-19b0-4580-bbe7-fb12f1edbcce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:03.024716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dfc68e-19b0-4580-bbe7-fb12f1edbcce None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:03.036597 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 393/815] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:03.037388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfc68e-19b0-4580-bbe7-fb12f1edbcce None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:03.092020 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ff38e-b4aa-4265-8529-bf9324f60a1a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:03.096806 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:03.127410 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03cd8c-ccc1-46b1-848d-180812c8d89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:03.128111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03cd8c-ccc1-46b1-848d-180812c8d89d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:03.128397 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03cd8c-ccc1-46b1-848d-180812c8d89d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:03.128677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03cd8c-ccc1-46b1-848d-180812c8d89d None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:03.129465 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03cd8c-ccc1-46b1-848d-180812c8d89d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:03.129776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03cd8c-ccc1-46b1-848d-180812c8d89d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:03.131228 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03cd8c-ccc1-46b1-848d-180812c8d89d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:03.138221 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfc68e-19b0-4580-bbe7-fb12f1edbcce None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:03.139844 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:03.144258 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 423/816] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:04:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:03.145557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03cd8c-ccc1-46b1-848d-180812c8d89d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:03.181322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57090694-d504-47bb-b6e4-d56052031e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:03.182122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57090694-d504-47bb-b6e4-d56052031e76 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:03.182395 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57090694-d504-47bb-b6e4-d56052031e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:03.182673 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57090694-d504-47bb-b6e4-d56052031e76 None admin] PATH_INFO: `/v3/services/network` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:03.183301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57090694-d504-47bb-b6e4-d56052031e76 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:03.184735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57090694-d504-47bb-b6e4-d56052031e76 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:03.196519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-57090694-d504-47bb-b6e4-d56052031e76 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 21:04:03.196519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:03.196519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:03.196519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:03.196519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:03.196519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:03.196519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:03.196519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:03.196519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 21:04:03.196519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:03.196519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:03.196519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:03.196519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:03.196519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:03.196519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:03.196519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:03.196519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:03.197507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:03.197507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:03.197507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:03.197507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:03.197507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:03.197507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:03.197507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:03.197507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:03.197507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:03.197507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:03.197507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:03.197507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:03.197507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:03.197507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:03.197507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:03.197507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:03.198383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:03.198383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:03.198383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:03.198383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:03.198383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:03.198383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:03.198383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:03.198383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:03.198383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:03.198383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:03.198383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:03.198383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:03.198383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 21:04:03.198383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:03.198383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 394/817] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:04:03 2022] GET /identity/v3/services/network => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:03.199206 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57090694-d504-47bb-b6e4-d56052031e76 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:03.254133 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03cd8c-ccc1-46b1-848d-180812c8d89d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:03.287060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2b896-6412-45fe-af6c-dd2de5c823a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:03.287749 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2b896-6412-45fe-af6c-dd2de5c823a6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:03.288034 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2b896-6412-45fe-af6c-dd2de5c823a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:03.288313 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2b896-6412-45fe-af6c-dd2de5c823a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:03.289084 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2b896-6412-45fe-af6c-dd2de5c823a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:03.289404 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2b896-6412-45fe-af6c-dd2de5c823a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:03.290829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2b896-6412-45fe-af6c-dd2de5c823a6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:03.300052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57090694-d504-47bb-b6e4-d56052031e76 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:03.302693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 424/818] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:04:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:03.303291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2b896-6412-45fe-af6c-dd2de5c823a6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:03.323924 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:03.336684 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e0482-b957-4392-893a-16884f8daa06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:03.337350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e0482-b957-4392-893a-16884f8daa06 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:03.337631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e0482-b957-4392-893a-16884f8daa06 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:03.337893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e0482-b957-4392-893a-16884f8daa06 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:03.338631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e0482-b957-4392-893a-16884f8daa06 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:03.338938 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e0482-b957-4392-893a-16884f8daa06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:03.340341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e0482-b957-4392-893a-16884f8daa06 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:03.345478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 395/819] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Jun 27 21:04:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:03.346157 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e0482-b957-4392-893a-16884f8daa06 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:03.365650 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:03.406434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2b896-6412-45fe-af6c-dd2de5c823a6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:03.441171 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a889cf-14bc-4f86-b115-5c66b1f46280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:03.441968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a889cf-14bc-4f86-b115-5c66b1f46280 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:03.442247 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a889cf-14bc-4f86-b115-5c66b1f46280 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:03.442617 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a889cf-14bc-4f86-b115-5c66b1f46280 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:03.446776 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e0482-b957-4392-893a-16884f8daa06 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:03.450072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a889cf-14bc-4f86-b115-5c66b1f46280 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:03.451524 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a889cf-14bc-4f86-b115-5c66b1f46280 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:03.465883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-44a889cf-14bc-4f86-b115-5c66b1f46280 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:04:03.465883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:03.465883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:03.465883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:03.465883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:03.465883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:03.465883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:03.465883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:03.465883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:03.465883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:03.465883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:03.465883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:03.465883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:04:03.465883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:04:03.465883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:04:03.465883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:04:03.466795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:03.466795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:03.466795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:04:03.466795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:03.466795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:04:03.466795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:03.466795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:03.466795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:03.466795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:03.466795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:03.466795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:03.466795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:03.466795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:03.466795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:03.466795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:03.466795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:03.466795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:03.466795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:03.467757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:03.467757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:03.467757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:04:03.467757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:04:03.467757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:04:03.467757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:04:03.467757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:04:03.467757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:04:03.467757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:04:03.467757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:03.467757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 425/820] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:04:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:03.468332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a889cf-14bc-4f86-b115-5c66b1f46280 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:03.482028 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dcb16f-421b-4d2a-bd23-791cc3edd7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:03.482930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcb16f-421b-4d2a-bd23-791cc3edd7e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:03.483212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcb16f-421b-4d2a-bd23-791cc3edd7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:03.483489 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcb16f-421b-4d2a-bd23-791cc3edd7e5 None admin] PATH_INFO: `/v3/projects/dfc29b021d1646819137c89e93f33601/users/e802da0b078e457da466a9a7b0f15085/roles/fb120deba4284112b12de1a251b7da76` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:03.497980 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dcb16f-421b-4d2a-bd23-791cc3edd7e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc29b021d1646819137c89e93f33601, user_id=e802da0b078e457da466a9a7b0f15085, role_id=fb120deba4284112b12de1a251b7da76)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:03.499439 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dcb16f-421b-4d2a-bd23-791cc3edd7e5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:03.512661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 396/821] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:04:03 2022] PUT /identity/v3/projects/dfc29b021d1646819137c89e93f33601/users/e802da0b078e457da466a9a7b0f15085/roles/fb120deba4284112b12de1a251b7da76 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:04:03.513443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dcb16f-421b-4d2a-bd23-791cc3edd7e5 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:03.564681 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:03.580782 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a889cf-14bc-4f86-b115-5c66b1f46280 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:03.607981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:03.624730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f924a32-ebf3-4e7b-ac50-e18f1ee4661c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:03.625419 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f924a32-ebf3-4e7b-ac50-e18f1ee4661c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:03.625703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f924a32-ebf3-4e7b-ac50-e18f1ee4661c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:03.626053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f924a32-ebf3-4e7b-ac50-e18f1ee4661c None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:03.626891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f924a32-ebf3-4e7b-ac50-e18f1ee4661c None admin] RBAC: Adding query filter params (name=network) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:03.627209 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f924a32-ebf3-4e7b-ac50-e18f1ee4661c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:03.628682 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f924a32-ebf3-4e7b-ac50-e18f1ee4661c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:03.639170 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 426/822] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Jun 27 21:04:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:03.639170 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f924a32-ebf3-4e7b-ac50-e18f1ee4661c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:03.645501 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dcb16f-421b-4d2a-bd23-791cc3edd7e5 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:03.688137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a79cb-981d-4dd0-beed-704f30b84f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:03.688837 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a79cb-981d-4dd0-beed-704f30b84f8a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:03.689121 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a79cb-981d-4dd0-beed-704f30b84f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:03.689681 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a79cb-981d-4dd0-beed-704f30b84f8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:03.690520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a79cb-981d-4dd0-beed-704f30b84f8a None admin] RBAC: Adding query filter params (role.id=5c3c095bb53943eca95c4e049cd8cbf5, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=8d7fe01ef64f46f6a0e77d5945219b33) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:03.690832 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a79cb-981d-4dd0-beed-704f30b84f8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:03.692373 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a79cb-981d-4dd0-beed-704f30b84f8a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:03.699379 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 397/823] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:04:03 2022] GET /identity/v3/role_assignments?user.id=8d7fe01ef64f46f6a0e77d5945219b33&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=5c3c095bb53943eca95c4e049cd8cbf5 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:03.700220 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a79cb-981d-4dd0-beed-704f30b84f8a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:03.761777 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f924a32-ebf3-4e7b-ac50-e18f1ee4661c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:03.802672 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd34f8-41c7-4430-b8f2-165153b27219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:03.803366 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd34f8-41c7-4430-b8f2-165153b27219 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:03.803689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd34f8-41c7-4430-b8f2-165153b27219 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:03.803970 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd34f8-41c7-4430-b8f2-165153b27219 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:03.804729 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd34f8-41c7-4430-b8f2-165153b27219 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:03.806390 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd34f8-41c7-4430-b8f2-165153b27219 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:03.834039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 427/824] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Jun 27 21:04:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:03.835261 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd34f8-41c7-4430-b8f2-165153b27219 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:03.836047 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:03.839849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a79cb-981d-4dd0-beed-704f30b84f8a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:03.897340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5182155-2975-40e7-b8b7-60cd5e95c6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:03.897957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5182155-2975-40e7-b8b7-60cd5e95c6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:03.898240 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5182155-2975-40e7-b8b7-60cd5e95c6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:03.898515 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5182155-2975-40e7-b8b7-60cd5e95c6e9 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:03.899891 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5182155-2975-40e7-b8b7-60cd5e95c6e9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:03.900312 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5182155-2975-40e7-b8b7-60cd5e95c6e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:03.902158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5182155-2975-40e7-b8b7-60cd5e95c6e9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:03.908488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:03.935143 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 398/825] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Jun 27 21:04:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:03.944748 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5182155-2975-40e7-b8b7-60cd5e95c6e9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:03.995351 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd34f8-41c7-4430-b8f2-165153b27219 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:04.037836 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc0e8c-3aeb-413d-94fe-834204d62a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:04.039377 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc0e8c-3aeb-413d-94fe-834204d62a02 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:04.039693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc0e8c-3aeb-413d-94fe-834204d62a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:04.039975 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc0e8c-3aeb-413d-94fe-834204d62a02 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:04.040757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc0e8c-3aeb-413d-94fe-834204d62a02 None admin] RBAC: Adding query filter params (type=network) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:04.041163 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc0e8c-3aeb-413d-94fe-834204d62a02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:04.042499 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc0e8c-3aeb-413d-94fe-834204d62a02 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:04.050413 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 428/826] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Jun 27 21:04:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:04.056859 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc0e8c-3aeb-413d-94fe-834204d62a02 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:04.086715 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5182155-2975-40e7-b8b7-60cd5e95c6e9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:04.137035 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24cbc79-ddc9-48cf-8b28-b072f4826dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:04.137992 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cbc79-ddc9-48cf-8b28-b072f4826dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:04.138371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cbc79-ddc9-48cf-8b28-b072f4826dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:04.139129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cbc79-ddc9-48cf-8b28-b072f4826dd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:04.151033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24cbc79-ddc9-48cf-8b28-b072f4826dd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:04.152612 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24cbc79-ddc9-48cf-8b28-b072f4826dd4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:04.172174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-a24cbc79-ddc9-48cf-8b28-b072f4826dd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:04.172174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:04.172174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:04.172174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:04.172174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:04.172174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:04.172174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:04.172174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:04.172174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:04.172174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:04.172174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:04.172174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:04.172174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:04.172174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:04.172174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:04.173355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:04.173355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:04.173355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:04.173355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:04.173355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:04.173355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:04.173355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:04.173355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:04.173355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:04.173355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:04.173355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:04.173355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:04.173355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:04.173355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:04.173355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:04.173355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:04.173355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:04.174300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:04.174300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:04.174300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:04.174300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:04.175327 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 399/827] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:04.190598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc0e8c-3aeb-413d-94fe-834204d62a02 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:04.193418 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:04.196652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24cbc79-ddc9-48cf-8b28-b072f4826dd4 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:04.210271 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:04.233249 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe45d0f-e5db-4179-a3f7-78b51851d1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:04.234266 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe45d0f-e5db-4179-a3f7-78b51851d1be None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:04.234732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe45d0f-e5db-4179-a3f7-78b51851d1be None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:04.235184 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe45d0f-e5db-4179-a3f7-78b51851d1be None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:04.236218 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe45d0f-e5db-4179-a3f7-78b51851d1be None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:04.238060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe45d0f-e5db-4179-a3f7-78b51851d1be None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:04.263100 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 429/828] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Jun 27 21:04:04 2022] POST /identity/v3/endpoints => generated 327 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:04.410928 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24cbc79-ddc9-48cf-8b28-b072f4826dd4 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:04.463252 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ed4e5-d844-40cf-b3e1-62394bd57c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:04.463970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ed4e5-d844-40cf-b3e1-62394bd57c7a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:04.464254 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ed4e5-d844-40cf-b3e1-62394bd57c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:04.464528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ed4e5-d844-40cf-b3e1-62394bd57c7a None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:04.465273 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ed4e5-d844-40cf-b3e1-62394bd57c7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:04.465573 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ed4e5-d844-40cf-b3e1-62394bd57c7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:04.466984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ed4e5-d844-40cf-b3e1-62394bd57c7a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:04.480504 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 400/829] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:04.488170 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe45d0f-e5db-4179-a3f7-78b51851d1be None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:04.599709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe45d0f-e5db-4179-a3f7-78b51851d1be None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:04.634361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d611330-55cd-4e3a-92fd-7a7d602f5715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:04.635524 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d611330-55cd-4e3a-92fd-7a7d602f5715 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:04.636073 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d611330-55cd-4e3a-92fd-7a7d602f5715 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:04.636549 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d611330-55cd-4e3a-92fd-7a7d602f5715 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:04.643919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d611330-55cd-4e3a-92fd-7a7d602f5715 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:04.645655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d611330-55cd-4e3a-92fd-7a7d602f5715 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:04.661101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4d611330-55cd-4e3a-92fd-7a7d602f5715 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:04.661101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:04.661101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:04.661101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:04.661101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:04.661101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:04.661101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:04.661101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:04.661101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:04.661101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:04.661101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:04.661101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:04.661101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:04:04.661101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:04:04.661101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:04:04.662083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:04:04.662083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:04.662083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:04.662083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:04.662083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:04.662083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:04.662083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:04.662083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:04.662083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:04.662083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:04.662083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:04.662083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:04.662083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:04.662083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:04.662083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:04.662083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:04.662083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:04.662959 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:04:04.662959 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:04:04.662959 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:04:04.662959 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:04:04.662959 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:04:04.662959 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:04:04.662959 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:04.662959 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:04.665489 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 430/830] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:04:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:04.671391 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ed4e5-d844-40cf-b3e1-62394bd57c7a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:04.743441 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:04.796295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ed4e5-d844-40cf-b3e1-62394bd57c7a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:04.840297 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a32d4-ec23-4eca-b236-dc01256f63d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:04.840977 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a32d4-ec23-4eca-b236-dc01256f63d4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:04.841258 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a32d4-ec23-4eca-b236-dc01256f63d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:04.841533 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a32d4-ec23-4eca-b236-dc01256f63d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:04.842327 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a32d4-ec23-4eca-b236-dc01256f63d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:04.842634 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a32d4-ec23-4eca-b236-dc01256f63d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:04.844115 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a32d4-ec23-4eca-b236-dc01256f63d4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:04.861713 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 401/831] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:04:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:04.867332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d611330-55cd-4e3a-92fd-7a7d602f5715 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:04.890000 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:04.969225 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d611330-55cd-4e3a-92fd-7a7d602f5715 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:05.004295 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02989c8-efd9-42c3-baef-a024be2e84f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:05.005217 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02989c8-efd9-42c3-baef-a024be2e84f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:05.005500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02989c8-efd9-42c3-baef-a024be2e84f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:05.005776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02989c8-efd9-42c3-baef-a024be2e84f4 None admin] PATH_INFO: `/v3/projects/dfc29b021d1646819137c89e93f33601/users/f28acb0322114aebafcf62bdf9a346df/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:05.007776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02989c8-efd9-42c3-baef-a024be2e84f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc29b021d1646819137c89e93f33601, user_id=f28acb0322114aebafcf62bdf9a346df, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:05.009240 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02989c8-efd9-42c3-baef-a024be2e84f4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:05.025412 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 431/832] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:04:04 2022] PUT /identity/v3/projects/dfc29b021d1646819137c89e93f33601/users/f28acb0322114aebafcf62bdf9a346df/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:04:05.681990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5214a57-e83f-405d-b0ef-59ae288bf3c2 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:05.682518 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5214a57-e83f-405d-b0ef-59ae288bf3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:05.682518 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5214a57-e83f-405d-b0ef-59ae288bf3c2 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:05.684207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 402/833] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:04:05.689477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11c851-bd52-4028-940e-bf2c2cb833d5 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:05.690124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11c851-bd52-4028-940e-bf2c2cb833d5 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:05.690124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11c851-bd52-4028-940e-bf2c2cb833d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:05.709131 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2a11c851-bd52-4028-940e-bf2c2cb833d5 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:05.710355 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2a11c851-bd52-4028-940e-bf2c2cb833d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:05.777979 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 432/834] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:05.801883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5214a57-e83f-405d-b0ef-59ae288bf3c2 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:06.032279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5214a57-e83f-405d-b0ef-59ae288bf3c2 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:06.056228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:06.068099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57893dc1-9471-42be-adc1-084dfcbef840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:06.068920 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57893dc1-9471-42be-adc1-084dfcbef840 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:06.069201 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57893dc1-9471-42be-adc1-084dfcbef840 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:06.069474 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57893dc1-9471-42be-adc1-084dfcbef840 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:06.080305 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57893dc1-9471-42be-adc1-084dfcbef840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:06.088499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-57893dc1-9471-42be-adc1-084dfcbef840 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:04:06.088499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:06.088499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:06.088499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:06.088499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:06.088499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:06.088499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:06.088499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:06.088499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:06.088499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:06.088499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:06.088499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:06.088499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:06.088499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:06.088499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:06.088499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:06.089402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:06.089402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:06.089402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:06.089402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:06.089402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:06.089402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:06.089402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:06.089402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:06.089402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:06.089402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:06.089402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:06.089402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:06.089402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:06.089402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:06.089402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:06.089402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:06.090626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:06.090626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:06.090626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:06.090626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:06.090626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:06.090626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:06.090626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:06.090626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:06.090626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:06.090626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:06.090626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:04:06.090626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:06.090626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 403/835] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:04:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:06.095946 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11c851-bd52-4028-940e-bf2c2cb833d5 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:06.100516 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541abc38-4654-4caa-b5d0-11758a7bfdc6 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:06.100794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541abc38-4654-4caa-b5d0-11758a7bfdc6 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:06.101069 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541abc38-4654-4caa-b5d0-11758a7bfdc6 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:06.102671 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 404/836] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:04:06.107770 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bc01c-edfe-4ff1-92a9-d4d5450fd82a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:06.108056 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bc01c-edfe-4ff1-92a9-d4d5450fd82a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:06.108566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:06.109286 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bc01c-edfe-4ff1-92a9-d4d5450fd82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:06.128787 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-4a2bc01c-edfe-4ff1-92a9-d4d5450fd82a None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:06.129993 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4a2bc01c-edfe-4ff1-92a9-d4d5450fd82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:06.204948 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11c851-bd52-4028-940e-bf2c2cb833d5 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:06.210649 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 405/837] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:06.211977 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757594c-8c7b-4341-88b5-6325c8e64538 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:06.212238 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757594c-8c7b-4341-88b5-6325c8e64538 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:06.212539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757594c-8c7b-4341-88b5-6325c8e64538 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:06.213964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 406/838] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:06.219147 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04bb2a-5f72-42a2-a16f-d7c5a5092f1b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:06.219444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04bb2a-5f72-42a2-a16f-d7c5a5092f1b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:06.219807 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04bb2a-5f72-42a2-a16f-d7c5a5092f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:06.241205 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-dc04bb2a-5f72-42a2-a16f-d7c5a5092f1b None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:06.242635 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-dc04bb2a-5f72-42a2-a16f-d7c5a5092f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:06.254802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc1843-71c5-46c2-aba7-a8d51e9c4470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:06.255430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc1843-71c5-46c2-aba7-a8d51e9c4470 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:06.255745 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc1843-71c5-46c2-aba7-a8d51e9c4470 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:06.256022 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc1843-71c5-46c2-aba7-a8d51e9c4470 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:06.256781 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc1843-71c5-46c2-aba7-a8d51e9c4470 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:06.257092 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc1843-71c5-46c2-aba7-a8d51e9c4470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:06.258487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc1843-71c5-46c2-aba7-a8d51e9c4470 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:06.266769 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 433/839] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:04:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:06.267559 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc1843-71c5-46c2-aba7-a8d51e9c4470 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:06.319771 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 407/840] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:06.319771 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04bb2a-5f72-42a2-a16f-d7c5a5092f1b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:06.351216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:06.366577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:06.384279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc1843-71c5-46c2-aba7-a8d51e9c4470 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:06.418324 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf74ca-9c20-4dff-8c0a-f3e4d272564b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:06.419090 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf74ca-9c20-4dff-8c0a-f3e4d272564b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:06.419373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf74ca-9c20-4dff-8c0a-f3e4d272564b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:06.419684 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf74ca-9c20-4dff-8c0a-f3e4d272564b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:06.420338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf74ca-9c20-4dff-8c0a-f3e4d272564b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:06.421746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf74ca-9c20-4dff-8c0a-f3e4d272564b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:06.426358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04bb2a-5f72-42a2-a16f-d7c5a5092f1b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:06.439407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-deaf74ca-9c20-4dff-8c0a-f3e4d272564b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 21:04:06.439407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:06.439407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:06.439407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:06.439407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:06.439407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:06.439407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:06.439407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:06.439407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 21:04:06.439407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:06.439407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:06.439407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:06.439407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:06.439407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:06.439407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:06.439407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:06.440556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:06.440556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:06.440556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:06.440556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:06.440556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:06.440556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:06.440556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:06.440556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:06.440556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:06.440556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:06.440556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:06.440556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:06.440556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:06.440556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:06.440556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:06.440556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:06.441381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:06.441381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:06.441381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:06.441381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:06.441381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:06.441381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:06.441381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:06.441381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:06.441381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:06.441381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:06.441381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:06.441381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:06.441381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:06.441381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 21:04:06.441381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:06.441381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 434/841] 10.222.0.27 () {56 vars in 1132 bytes} [Mon Jun 27 21:04:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:06.443154 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf74ca-9c20-4dff-8c0a-f3e4d272564b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:06.468620 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c234072-5e09-4437-b7f4-9d2201f593b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:06.469432 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c234072-5e09-4437-b7f4-9d2201f593b8 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:06.469781 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c234072-5e09-4437-b7f4-9d2201f593b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:06.470065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c234072-5e09-4437-b7f4-9d2201f593b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:06.481047 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c234072-5e09-4437-b7f4-9d2201f593b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:06.482526 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c234072-5e09-4437-b7f4-9d2201f593b8 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:06.500342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-8c234072-5e09-4437-b7f4-9d2201f593b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:06.500342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:06.500342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:06.500342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:06.500342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:06.500342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:06.500342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:06.500342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:06.500342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:06.500342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:06.500342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:06.500342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:06.500342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:06.500342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:06.500342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:06.501368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:06.501368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:06.501368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:06.501368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:06.501368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:06.501368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:06.501368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:06.501368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:06.501368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:06.501368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:06.501368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:06.501368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:06.501368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:06.501368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:06.501368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:06.501368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:06.501368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:06.502268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:06.502268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:06.502268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:06.502268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:06.502268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 408/842] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:06.502535 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c234072-5e09-4437-b7f4-9d2201f593b8 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:06.551027 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf74ca-9c20-4dff-8c0a-f3e4d272564b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:06.587178 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1195a0-7fb0-4078-8db3-704e407bdc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:06.588159 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1195a0-7fb0-4078-8db3-704e407bdc0f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:06.588519 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1195a0-7fb0-4078-8db3-704e407bdc0f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:06.588796 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1195a0-7fb0-4078-8db3-704e407bdc0f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:06.589447 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1195a0-7fb0-4078-8db3-704e407bdc0f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:06.591080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1195a0-7fb0-4078-8db3-704e407bdc0f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:06.594381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:06.602410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:06.607363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-6f1195a0-7fb0-4078-8db3-704e407bdc0f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 21:04:06.607363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:06.607363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:06.607363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:06.607363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:06.607363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:06.607363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:06.607363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:06.607363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 21:04:06.607363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:06.607363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:06.607363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:06.607363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:06.607363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:06.607363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:06.607363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:06.607363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:06.608385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:06.608385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:06.608385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:06.608385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:06.608385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:06.608385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:06.608385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:06.608385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:06.608385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:06.608385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:06.608385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:06.608385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:06.608385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:06.608385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:06.608385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:06.608385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:06.609202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:06.609202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:06.609202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:06.609202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:06.609202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:06.609202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:06.609202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:06.609202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:06.609202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:06.609202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:06.609202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:06.609202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:06.609202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 21:04:06.609202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:06.609202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 435/843] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:04:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:06.609974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1195a0-7fb0-4078-8db3-704e407bdc0f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:06.615811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c234072-5e09-4437-b7f4-9d2201f593b8 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:06.649626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650efb9b-079a-456c-8801-81b9c10483f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:06.650308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650efb9b-079a-456c-8801-81b9c10483f9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:06.650595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650efb9b-079a-456c-8801-81b9c10483f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:06.650870 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650efb9b-079a-456c-8801-81b9c10483f9 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:06.651570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650efb9b-079a-456c-8801-81b9c10483f9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:06.651908 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650efb9b-079a-456c-8801-81b9c10483f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:06.653291 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650efb9b-079a-456c-8801-81b9c10483f9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:06.658547 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 409/844] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Jun 27 21:04:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:06.663697 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650efb9b-079a-456c-8801-81b9c10483f9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:06.713669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1195a0-7fb0-4078-8db3-704e407bdc0f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:06.748302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f644d7-487b-443a-9940-664bd5c2407b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:06.749022 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f644d7-487b-443a-9940-664bd5c2407b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:06.749306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f644d7-487b-443a-9940-664bd5c2407b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:06.749578 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f644d7-487b-443a-9940-664bd5c2407b None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:06.750297 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f644d7-487b-443a-9940-664bd5c2407b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:06.750601 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f644d7-487b-443a-9940-664bd5c2407b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:06.752046 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f644d7-487b-443a-9940-664bd5c2407b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:06.769042 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 436/845] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:06.769658 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650efb9b-079a-456c-8801-81b9c10483f9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:06.770095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f644d7-487b-443a-9940-664bd5c2407b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:06.804524 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d51572-6d5d-403b-8f42-e514784153eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:06.805112 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d51572-6d5d-403b-8f42-e514784153eb None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:06.805390 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d51572-6d5d-403b-8f42-e514784153eb None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:06.805659 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d51572-6d5d-403b-8f42-e514784153eb None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:06.806357 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d51572-6d5d-403b-8f42-e514784153eb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:06.806663 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d51572-6d5d-403b-8f42-e514784153eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:06.808063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d51572-6d5d-403b-8f42-e514784153eb None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:06.812461 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 410/846] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:04:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:06.813221 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d51572-6d5d-403b-8f42-e514784153eb None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:06.838092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:06.840844 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:06.884050 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f644d7-487b-443a-9940-664bd5c2407b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:06.917427 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d51572-6d5d-403b-8f42-e514784153eb None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:06.920814 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09322d-30fc-4836-b0ed-9bc09befb138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:06.921482 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09322d-30fc-4836-b0ed-9bc09befb138 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:06.921759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09322d-30fc-4836-b0ed-9bc09befb138 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:06.922037 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09322d-30fc-4836-b0ed-9bc09befb138 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:06.922771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09322d-30fc-4836-b0ed-9bc09befb138 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:06.923071 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09322d-30fc-4836-b0ed-9bc09befb138 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:06.924594 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09322d-30fc-4836-b0ed-9bc09befb138 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:06.939639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 437/847] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Jun 27 21:04:06 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:06.939639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09322d-30fc-4836-b0ed-9bc09befb138 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:06.968339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d24bb0-fab6-4af1-9e8f-15a2038cf6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:06.969487 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d24bb0-fab6-4af1-9e8f-15a2038cf6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:06.970274 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d24bb0-fab6-4af1-9e8f-15a2038cf6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:06.970562 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d24bb0-fab6-4af1-9e8f-15a2038cf6b0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:06.981948 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d24bb0-fab6-4af1-9e8f-15a2038cf6b0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:06.984109 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d24bb0-fab6-4af1-9e8f-15a2038cf6b0 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:07.002672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-a7d24bb0-fab6-4af1-9e8f-15a2038cf6b0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:04:07.002672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:07.002672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:07.002672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:07.002672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:07.002672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:07.002672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:07.002672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:07.002672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:07.002672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:07.002672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:07.002672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:07.002672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:04:07.002672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:04:07.002672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:04:07.002672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:04:07.003721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:07.003721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:07.003721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:04:07.003721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:07.003721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:04:07.003721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:07.003721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:07.003721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:07.003721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:07.003721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:07.003721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:07.003721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:07.003721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:07.003721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:07.003721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:07.003721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:07.003721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:07.003721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:07.004666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:07.004666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:07.004666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:04:07.004666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:04:07.004666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:04:07.004666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:04:07.004666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:04:07.004666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:04:07.004666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:04:07.004666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:07.008922 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 411/848] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Jun 27 21:04:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:07.009922 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d24bb0-fab6-4af1-9e8f-15a2038cf6b0 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:07.062957 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09322d-30fc-4836-b0ed-9bc09befb138 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:07.092780 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:07.104236 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c05523-a7b4-4d3d-8458-06bf2e7c65ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:07.104911 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c05523-a7b4-4d3d-8458-06bf2e7c65ed None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:07.105188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c05523-a7b4-4d3d-8458-06bf2e7c65ed None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:07.105961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c05523-a7b4-4d3d-8458-06bf2e7c65ed None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:07.106679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c05523-a7b4-4d3d-8458-06bf2e7c65ed None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:07.106985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c05523-a7b4-4d3d-8458-06bf2e7c65ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:07.108695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c05523-a7b4-4d3d-8458-06bf2e7c65ed None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:07.114306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 438/849] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:04:06 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:07.115676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2449ce8-0061-48f9-89ec-1c4e019565dd None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:07.115971 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2449ce8-0061-48f9-89ec-1c4e019565dd None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:07.116250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2449ce8-0061-48f9-89ec-1c4e019565dd None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:07.117653 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 439/850] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:07.119429 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:07.123485 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06e81d-f9c0-4212-a037-8cf92f1ef362 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:07.123859 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06e81d-f9c0-4212-a037-8cf92f1ef362 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:07.124133 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06e81d-f9c0-4212-a037-8cf92f1ef362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:07.143768 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d24bb0-fab6-4af1-9e8f-15a2038cf6b0 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:07.145687 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ab06e81d-f9c0-4212-a037-8cf92f1ef362 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:07.149141 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ab06e81d-f9c0-4212-a037-8cf92f1ef362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:07.192314 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a11312-a276-4309-9587-ae4ed037c7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:07.193259 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a11312-a276-4309-9587-ae4ed037c7fc None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:07.193591 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a11312-a276-4309-9587-ae4ed037c7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:07.194842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a11312-a276-4309-9587-ae4ed037c7fc None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:07.195843 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a11312-a276-4309-9587-ae4ed037c7fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:07.196163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a11312-a276-4309-9587-ae4ed037c7fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:07.197827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a11312-a276-4309-9587-ae4ed037c7fc None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:07.222288 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 412/851] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Jun 27 21:04:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:07.227873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a11312-a276-4309-9587-ae4ed037c7fc None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:07.240568 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 440/852] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:07.263977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06e81d-f9c0-4212-a037-8cf92f1ef362 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:07.350482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a11312-a276-4309-9587-ae4ed037c7fc None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:07.384744 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa3255-9600-498a-b92d-54771763355f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:07.385404 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa3255-9600-498a-b92d-54771763355f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:07.385684 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa3255-9600-498a-b92d-54771763355f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:07.385956 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa3255-9600-498a-b92d-54771763355f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:07.388132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06e81d-f9c0-4212-a037-8cf92f1ef362 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:07.428017 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:07.461145 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa3255-9600-498a-b92d-54771763355f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:07.462668 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa3255-9600-498a-b92d-54771763355f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:07.556538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:07.579233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 413/853] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:07 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:07.584244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd6dcc-f641-4239-9e12-5a625b2f4138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:07.584671 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa3255-9600-498a-b92d-54771763355f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:07.585259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd6dcc-f641-4239-9e12-5a625b2f4138 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:07.585635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd6dcc-f641-4239-9e12-5a625b2f4138 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:07.586007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd6dcc-f641-4239-9e12-5a625b2f4138 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:07.597695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd6dcc-f641-4239-9e12-5a625b2f4138 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:07.605655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-cecd6dcc-f641-4239-9e12-5a625b2f4138 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:07.605655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:07.605655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:07.605655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:07.605655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:07.605655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:07.605655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:07.605655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:07.605655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:07.605655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:07.605655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:07.605655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:07.605655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:07.605655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:07.605655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:07.605655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:07.606747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:07.606747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:07.606747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:07.606747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:07.606747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:07.606747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:07.606747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:07.606747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:07.606747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:07.606747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:07.606747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:07.606747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:07.606747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:07.606747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:07.606747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:07.606747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:07.607582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:07.607582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:07.607582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:07.607582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:07.607582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:07.607582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:07.607582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:07.607582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:07.607582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:07.607582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:07.607582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:07.607582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:07.608400 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 441/854] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:04:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:07.613639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd6dcc-f641-4239-9e12-5a625b2f4138 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:07.636283 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:07.708849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa3255-9600-498a-b92d-54771763355f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:07.728931 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd6dcc-f641-4239-9e12-5a625b2f4138 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:07.757250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f088e-6c47-4875-b1d1-deb581713ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:07.758062 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f088e-6c47-4875-b1d1-deb581713ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:07.758344 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f088e-6c47-4875-b1d1-deb581713ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:07.758618 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f088e-6c47-4875-b1d1-deb581713ec1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:07.769539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01611200-39c2-486f-b441-7fc15e0a845a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:07.770313 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01611200-39c2-486f-b441-7fc15e0a845a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:07.770596 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01611200-39c2-486f-b441-7fc15e0a845a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:07.770874 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01611200-39c2-486f-b441-7fc15e0a845a None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:07.771661 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01611200-39c2-486f-b441-7fc15e0a845a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:07.771989 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01611200-39c2-486f-b441-7fc15e0a845a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:07.772890 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f088e-6c47-4875-b1d1-deb581713ec1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:07.773404 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01611200-39c2-486f-b441-7fc15e0a845a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:07.775355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f088e-6c47-4875-b1d1-deb581713ec1 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:07.787223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 442/855] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:04:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:07.788240 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01611200-39c2-486f-b441-7fc15e0a845a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:07.796767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-729f088e-6c47-4875-b1d1-deb581713ec1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:07.796767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:07.796767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:07.796767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:07.796767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:07.796767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:07.796767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:07.796767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:07.796767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:07.796767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:07.796767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:07.796767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:07.796767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:07.796767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:07.796767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:07.797662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:07.797662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:07.797662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:07.797662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:07.797662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:07.797662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:07.797662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:07.797662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:07.797662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:07.797662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:07.797662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:07.797662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:07.797662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:07.797662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:07.797662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:07.797662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:07.797662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:07.798550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:07.798550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:07.798550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:07.798550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:07.805615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 414/856] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:07.806354 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f088e-6c47-4875-b1d1-deb581713ec1 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:07.842179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:07.906061 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01611200-39c2-486f-b441-7fc15e0a845a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:07.927689 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:07.941545 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a498fc2-679f-4b08-8510-8843083187a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:07.941924 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f088e-6c47-4875-b1d1-deb581713ec1 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:07.942359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a498fc2-679f-4b08-8510-8843083187a4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:07.942644 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a498fc2-679f-4b08-8510-8843083187a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:07.942918 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a498fc2-679f-4b08-8510-8843083187a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:07.954235 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a498fc2-679f-4b08-8510-8843083187a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:07.955744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a498fc2-679f-4b08-8510-8843083187a4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:07.973364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4a498fc2-679f-4b08-8510-8843083187a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:07.973364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:07.973364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:07.973364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:07.973364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:07.973364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:07.973364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:07.973364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:07.973364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:07.973364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:07.973364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:07.973364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:07.973364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:07.973364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:07.973364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:07.974248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:07.974248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:07.974248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:07.974248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:07.974248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:07.974248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:07.974248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:07.974248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:07.974248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:07.974248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:07.974248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:07.974248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:07.974248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:07.974248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:07.974248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:07.974248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:07.974248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:07.975112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:07.975112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:07.975112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:07.975112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:07.979261 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 443/857] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:07.980151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a498fc2-679f-4b08-8510-8843083187a4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:07.991812 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e8ddb-d6df-4314-9780-47ae661126f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:07.992527 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e8ddb-d6df-4314-9780-47ae661126f6 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:07.992809 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e8ddb-d6df-4314-9780-47ae661126f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:07.993093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e8ddb-d6df-4314-9780-47ae661126f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:07.993832 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e8ddb-d6df-4314-9780-47ae661126f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:07.994143 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e8ddb-d6df-4314-9780-47ae661126f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:07.995534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e8ddb-d6df-4314-9780-47ae661126f6 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:08.011491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 415/858] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:08.016779 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e8ddb-d6df-4314-9780-47ae661126f6 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:08.088178 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:08.098370 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a498fc2-679f-4b08-8510-8843083187a4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:08.120358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e8ddb-d6df-4314-9780-47ae661126f6 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:08.136832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99095c7-cbcd-41e7-b823-9db0b9a0e7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:08.137508 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99095c7-cbcd-41e7-b823-9db0b9a0e7fa None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:08.137789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99095c7-cbcd-41e7-b823-9db0b9a0e7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:08.138064 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99095c7-cbcd-41e7-b823-9db0b9a0e7fa None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:08.138792 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99095c7-cbcd-41e7-b823-9db0b9a0e7fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:08.139102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99095c7-cbcd-41e7-b823-9db0b9a0e7fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:08.140728 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99095c7-cbcd-41e7-b823-9db0b9a0e7fa None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:08.153999 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 444/859] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:08.159706 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c42e79-b9fe-4f40-9bc2-a78fbfdc3f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:08.160222 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c42e79-b9fe-4f40-9bc2-a78fbfdc3f0e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:08.160971 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99095c7-cbcd-41e7-b823-9db0b9a0e7fa None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:08.161498 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c42e79-b9fe-4f40-9bc2-a78fbfdc3f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:08.161768 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c42e79-b9fe-4f40-9bc2-a78fbfdc3f0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:08.170578 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c42e79-b9fe-4f40-9bc2-a78fbfdc3f0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:08.172633 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c42e79-b9fe-4f40-9bc2-a78fbfdc3f0e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:08.192958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-63c42e79-b9fe-4f40-9bc2-a78fbfdc3f0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:08.192958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:08.192958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:08.192958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:08.192958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:08.192958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:08.192958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:08.192958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:08.192958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:08.192958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:08.192958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:08.192958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:08.192958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:04:08.192958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:04:08.192958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:04:08.193930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:04:08.193930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:08.193930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:08.193930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:08.193930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:08.193930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:08.193930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:08.193930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:08.193930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:08.193930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:08.193930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:08.193930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:08.193930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:08.193930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:08.193930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:08.193930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:08.193930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:08.194850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:04:08.194850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:04:08.194850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:04:08.194850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:04:08.194850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:04:08.194850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:04:08.194850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:08.194850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:08.194850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 416/860] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:04:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:08.200758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c42e79-b9fe-4f40-9bc2-a78fbfdc3f0e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:08.219856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:08.275202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99095c7-cbcd-41e7-b823-9db0b9a0e7fa None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:08.310785 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85c4e2-28ee-4676-9d2e-1868e3e616ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:08.311594 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85c4e2-28ee-4676-9d2e-1868e3e616ed None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:08.311909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85c4e2-28ee-4676-9d2e-1868e3e616ed None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:08.312179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85c4e2-28ee-4676-9d2e-1868e3e616ed None admin] PATH_INFO: `/v3/users/swift` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:08.319724 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c42e79-b9fe-4f40-9bc2-a78fbfdc3f0e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:08.320075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85c4e2-28ee-4676-9d2e-1868e3e616ed None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:08.321575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85c4e2-28ee-4676-9d2e-1868e3e616ed None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:08.347082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-0a85c4e2-28ee-4676-9d2e-1868e3e616ed None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:04:08.347082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:08.347082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:08.347082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:08.347082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:08.347082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:08.347082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:08.347082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:08.347082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:08.347082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:08.347082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:08.347082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:08.347082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:04:08.347082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:04:08.347082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:04:08.347082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:04:08.348241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:08.348241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:08.348241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:04:08.348241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:08.348241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:04:08.348241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:08.348241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:08.348241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:08.348241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:08.348241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:08.348241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:08.348241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:08.348241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:08.348241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:08.348241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:08.348241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:08.348241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:08.348241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:08.349586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:08.349586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:08.349586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:04:08.349586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:04:08.349586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:04:08.349586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:04:08.349586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:04:08.349586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:04:08.349586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:04:08.349586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:08.349586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 445/861] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:04:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:08.354698 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85c4e2-28ee-4676-9d2e-1868e3e616ed None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:08.365502 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3028b1-348b-4859-abba-377498b1c61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:08.366181 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3028b1-348b-4859-abba-377498b1c61a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:08.366463 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3028b1-348b-4859-abba-377498b1c61a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:08.366746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3028b1-348b-4859-abba-377498b1c61a None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:08.367535 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3028b1-348b-4859-abba-377498b1c61a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:08.367911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3028b1-348b-4859-abba-377498b1c61a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:08.369332 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3028b1-348b-4859-abba-377498b1c61a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:08.375447 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:08.382329 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 417/862] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:04:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:08.387693 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3028b1-348b-4859-abba-377498b1c61a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:08.461270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:08.496173 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3028b1-348b-4859-abba-377498b1c61a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:08.498121 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85c4e2-28ee-4676-9d2e-1868e3e616ed None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:08.531188 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b2edc-4a10-4f46-9376-b3398bbb0752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:08.531915 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b2edc-4a10-4f46-9376-b3398bbb0752 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:08.532258 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b2edc-4a10-4f46-9376-b3398bbb0752 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:08.532505 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b2edc-4a10-4f46-9376-b3398bbb0752 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:08.533682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b2edc-4a10-4f46-9376-b3398bbb0752 None admin] RBAC: Adding query filter params (role.id=fb120deba4284112b12de1a251b7da76, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=e802da0b078e457da466a9a7b0f15085) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:08.533987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b2edc-4a10-4f46-9376-b3398bbb0752 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:08.535432 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b2edc-4a10-4f46-9376-b3398bbb0752 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:08.544684 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9f353-99ae-4d1e-86f8-52392e7a2d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:08.545668 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9f353-99ae-4d1e-86f8-52392e7a2d23 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:08.546156 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9f353-99ae-4d1e-86f8-52392e7a2d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:08.546637 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9f353-99ae-4d1e-86f8-52392e7a2d23 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:08.547865 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9f353-99ae-4d1e-86f8-52392e7a2d23 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:08.548362 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 418/863] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:04:08 2022] GET /identity/v3/role_assignments?user.id=e802da0b078e457da466a9a7b0f15085&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=fb120deba4284112b12de1a251b7da76 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:08.548589 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9f353-99ae-4d1e-86f8-52392e7a2d23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:08.550293 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9f353-99ae-4d1e-86f8-52392e7a2d23 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:08.573793 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 446/864] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Jun 27 21:04:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:08.581744 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b2edc-4a10-4f46-9376-b3398bbb0752 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:08.703953 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b2edc-4a10-4f46-9376-b3398bbb0752 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:08.741353 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7fc04-8e4d-48fb-8457-1272549397d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:08.742368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7fc04-8e4d-48fb-8457-1272549397d3 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:08.742852 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7fc04-8e4d-48fb-8457-1272549397d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:08.743308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7fc04-8e4d-48fb-8457-1272549397d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:08.747989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:08.816818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7fc04-8e4d-48fb-8457-1272549397d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:08.818361 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7fc04-8e4d-48fb-8457-1272549397d3 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:08.924598 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 419/865] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:08 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:08.930325 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9f353-99ae-4d1e-86f8-52392e7a2d23 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:09.005939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:09.038727 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9f353-99ae-4d1e-86f8-52392e7a2d23 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:09.064482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea1d11-6fa1-4b5a-b4de-d4592eb55422 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.064811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea1d11-6fa1-4b5a-b4de-d4592eb55422 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.065049 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea1d11-6fa1-4b5a-b4de-d4592eb55422 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:09.066698 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 420/866] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:09.071417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd4121-0c59-427f-8ad8-4b6d6ddda584 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.071737 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd4121-0c59-427f-8ad8-4b6d6ddda584 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.072018 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd4121-0c59-427f-8ad8-4b6d6ddda584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:09.074323 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd77f3-2a19-4b0a-8156-972e2529968b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:09.075147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd77f3-2a19-4b0a-8156-972e2529968b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.075524 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd77f3-2a19-4b0a-8156-972e2529968b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.075827 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd77f3-2a19-4b0a-8156-972e2529968b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:09.086370 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd77f3-2a19-4b0a-8156-972e2529968b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:09.087874 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd77f3-2a19-4b0a-8156-972e2529968b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:09.089178 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ffcd4121-0c59-427f-8ad8-4b6d6ddda584 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:09.090629 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ffcd4121-0c59-427f-8ad8-4b6d6ddda584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:09.105944 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4ebd77f3-2a19-4b0a-8156-972e2529968b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:09.105944 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:09.105944 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:09.105944 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:09.105944 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:09.105944 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:09.105944 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:09.105944 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:09.105944 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:09.105944 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:09.105944 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:09.105944 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:09.105944 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:09.105944 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:09.105944 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:09.106872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:09.106872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:09.106872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:09.106872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:09.106872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:09.106872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:09.106872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:09.106872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:09.106872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:09.106872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:09.106872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:09.106872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:09.106872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:09.106872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:09.106872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:09.106872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:09.106872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:09.107750 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:09.107750 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:09.107750 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:09.107750 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:09.107750 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 447/867] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:09.112869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd77f3-2a19-4b0a-8156-972e2529968b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:09.125483 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:09.169180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 421/868] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:09.192711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd4121-0c59-427f-8ad8-4b6d6ddda584 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:09.220302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd77f3-2a19-4b0a-8156-972e2529968b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:09.245437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:09.258023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a1381-6fc8-425e-a7cd-3e5deb26c40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:09.258730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a1381-6fc8-425e-a7cd-3e5deb26c40b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.259017 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a1381-6fc8-425e-a7cd-3e5deb26c40b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.259291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a1381-6fc8-425e-a7cd-3e5deb26c40b None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:09.260058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a1381-6fc8-425e-a7cd-3e5deb26c40b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:09.260370 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a1381-6fc8-425e-a7cd-3e5deb26c40b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:09.261766 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a1381-6fc8-425e-a7cd-3e5deb26c40b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:09.275290 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 448/869] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:09.275290 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e4487-9aba-4113-8e4e-1207030fcb94 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.275290 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e4487-9aba-4113-8e4e-1207030fcb94 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.275634 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e4487-9aba-4113-8e4e-1207030fcb94 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:09.280688 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 449/870] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:09.280688 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e4487-9aba-4113-8e4e-1207030fcb94 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:09.306039 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd4121-0c59-427f-8ad8-4b6d6ddda584 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:09.342464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ae982-37de-4b8c-a7b0-c55d5cb78dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:09.343140 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ae982-37de-4b8c-a7b0-c55d5cb78dc4 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.343417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ae982-37de-4b8c-a7b0-c55d5cb78dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.343721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ae982-37de-4b8c-a7b0-c55d5cb78dc4 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:09.344465 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ae982-37de-4b8c-a7b0-c55d5cb78dc4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:09.345897 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ae982-37de-4b8c-a7b0-c55d5cb78dc4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:09.362983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 422/871] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Jun 27 21:04:09 2022] POST /identity/v3/services => generated 269 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:04:09.367967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b52287-274f-4876-80eb-8b5c6080fd2d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.367967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b52287-274f-4876-80eb-8b5c6080fd2d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.367967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b52287-274f-4876-80eb-8b5c6080fd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:09.392945 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-64b52287-274f-4876-80eb-8b5c6080fd2d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:09.394201 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-64b52287-274f-4876-80eb-8b5c6080fd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:09.408174 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e4487-9aba-4113-8e4e-1207030fcb94 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:09.420853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:09.464211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f5cd5-5093-4be7-a4ab-9fc5c9898d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:09.465931 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f5cd5-5093-4be7-a4ab-9fc5c9898d02 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.466422 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f5cd5-5093-4be7-a4ab-9fc5c9898d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.467141 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f5cd5-5093-4be7-a4ab-9fc5c9898d02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:09.476415 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f5cd5-5093-4be7-a4ab-9fc5c9898d02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:09.477960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f5cd5-5093-4be7-a4ab-9fc5c9898d02 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:09.483458 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 423/872] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:09 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:09.496776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-331f5cd5-5093-4be7-a4ab-9fc5c9898d02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:09.496776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:09.496776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:09.496776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:09.496776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:09.496776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:09.496776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:09.496776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:09.496776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:09.496776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:09.496776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:09.496776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:09.496776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:04:09.496776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:04:09.496776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:04:09.497705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:04:09.497705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:09.497705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:09.497705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:09.497705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:09.497705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:09.497705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:09.497705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:09.497705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:09.497705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:09.497705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:09.497705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:09.497705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:09.497705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:09.497705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:09.497705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:09.497705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:09.498570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:04:09.498570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:04:09.498570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:04:09.498570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:04:09.498570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:04:09.498570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:04:09.498570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:09.498570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:09.498570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 450/873] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:04:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:09.504620 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b52287-274f-4876-80eb-8b5c6080fd2d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:09.506133 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f5cd5-5093-4be7-a4ab-9fc5c9898d02 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:09.557015 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:09.628313 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f5cd5-5093-4be7-a4ab-9fc5c9898d02 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:09.631942 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b52287-274f-4876-80eb-8b5c6080fd2d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:09.666241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed489c-6393-4feb-8740-26a85406180c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:09.667064 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed489c-6393-4feb-8740-26a85406180c None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.667677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed489c-6393-4feb-8740-26a85406180c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.667962 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed489c-6393-4feb-8740-26a85406180c None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:09.668936 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed489c-6393-4feb-8740-26a85406180c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:09.670315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed489c-6393-4feb-8740-26a85406180c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:09.672214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f073c-3c96-4a2d-a6e2-ee8710ec30d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:09.673529 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f073c-3c96-4a2d-a6e2-ee8710ec30d6 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.673825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f073c-3c96-4a2d-a6e2-ee8710ec30d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.674288 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f073c-3c96-4a2d-a6e2-ee8710ec30d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:09.675053 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f073c-3c96-4a2d-a6e2-ee8710ec30d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:09.675412 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f073c-3c96-4a2d-a6e2-ee8710ec30d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:09.677181 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f073c-3c96-4a2d-a6e2-ee8710ec30d6 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:09.686914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 451/874] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Jun 27 21:04:09 2022] POST /identity/v3/services => generated 251 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:09.691211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 424/875] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:04:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:09.697689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed489c-6393-4feb-8740-26a85406180c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:09.817383 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:09.828032 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed489c-6393-4feb-8740-26a85406180c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:09.869882 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff3b6c-eed2-4fe1-9a5a-dc626506aded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:09.870546 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff3b6c-eed2-4fe1-9a5a-dc626506aded None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.870821 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff3b6c-eed2-4fe1-9a5a-dc626506aded None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.871182 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff3b6c-eed2-4fe1-9a5a-dc626506aded None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:09.872045 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff3b6c-eed2-4fe1-9a5a-dc626506aded None admin] RBAC: Adding query filter params (role.id=5c3c095bb53943eca95c4e049cd8cbf5, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=f28acb0322114aebafcf62bdf9a346df) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:09.872354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff3b6c-eed2-4fe1-9a5a-dc626506aded None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:09.873804 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff3b6c-eed2-4fe1-9a5a-dc626506aded None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:09.880428 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 452/876] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:04:09 2022] GET /identity/v3/role_assignments?user.id=f28acb0322114aebafcf62bdf9a346df&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=5c3c095bb53943eca95c4e049cd8cbf5 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:10.771590 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6da56-a089-4963-a2e5-3a9a7ed01f44 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:10.772182 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6da56-a089-4963-a2e5-3a9a7ed01f44 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:10.772182 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6da56-a089-4963-a2e5-3a9a7ed01f44 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:10.773892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 425/877] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:10.779099 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaeb04e-98d0-4f38-8f73-3bf166d2e02a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:10.779509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaeb04e-98d0-4f38-8f73-3bf166d2e02a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:10.779705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaeb04e-98d0-4f38-8f73-3bf166d2e02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:10.797403 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-afaeb04e-98d0-4f38-8f73-3bf166d2e02a None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:10.798609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-afaeb04e-98d0-4f38-8f73-3bf166d2e02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:10.868308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 453/878] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:10.893290 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6da56-a089-4963-a2e5-3a9a7ed01f44 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:10.913188 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:11.000355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6da56-a089-4963-a2e5-3a9a7ed01f44 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:11.038045 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083e005-a7a5-411e-b74c-121b9b22696d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:11.038789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083e005-a7a5-411e-b74c-121b9b22696d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:11.039066 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083e005-a7a5-411e-b74c-121b9b22696d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:11.039337 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083e005-a7a5-411e-b74c-121b9b22696d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:11.050640 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2083e005-a7a5-411e-b74c-121b9b22696d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:11.058909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-2083e005-a7a5-411e-b74c-121b9b22696d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:04:11.058909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:11.058909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:11.058909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:11.058909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:11.058909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:11.058909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:11.058909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:11.058909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:11.058909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:11.058909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:11.058909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:11.058909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:11.058909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:11.058909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:11.058909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:11.059910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:11.059910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:11.059910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:11.059910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:11.059910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:11.059910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:11.059910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:11.059910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:11.059910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:11.059910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:11.059910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:11.059910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:11.059910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:11.059910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:11.059910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:11.059910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:11.060964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:11.060964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:11.060964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:11.060964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:11.060964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:11.060964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:11.060964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:11.060964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:11.060964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:11.060964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:11.060964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:04:11.060964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:11.060964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 426/879] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Jun 27 21:04:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:11.066318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaeb04e-98d0-4f38-8f73-3bf166d2e02a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:11.154329 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:11.166660 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaeb04e-98d0-4f38-8f73-3bf166d2e02a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:11.200011 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9a187-694e-465c-a380-3aaaa801c41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:11.200687 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9a187-694e-465c-a380-3aaaa801c41d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:11.200963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9a187-694e-465c-a380-3aaaa801c41d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:11.201233 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9a187-694e-465c-a380-3aaaa801c41d None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:11.201977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9a187-694e-465c-a380-3aaaa801c41d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:11.202279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9a187-694e-465c-a380-3aaaa801c41d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:11.203770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9a187-694e-465c-a380-3aaaa801c41d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:11.212051 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 454/880] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Jun 27 21:04:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:11.217672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083e005-a7a5-411e-b74c-121b9b22696d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:11.295640 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:11.319314 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083e005-a7a5-411e-b74c-121b9b22696d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:11.354465 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d370856-7ac3-45f4-a672-581e222d0ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:11.355282 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d370856-7ac3-45f4-a672-581e222d0ace None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:11.355562 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d370856-7ac3-45f4-a672-581e222d0ace None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:11.355865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d370856-7ac3-45f4-a672-581e222d0ace None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:11.366928 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d370856-7ac3-45f4-a672-581e222d0ace None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:11.368421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d370856-7ac3-45f4-a672-581e222d0ace None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:11.386491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-8d370856-7ac3-45f4-a672-581e222d0ace None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:11.386491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:11.386491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:11.386491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:11.386491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:11.386491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:11.386491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:11.386491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:11.386491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:11.386491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:11.386491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:11.386491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:11.386491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:11.386491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:11.386491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:11.387408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:11.387408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:11.387408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:11.387408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:11.387408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:11.387408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:11.387408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:11.387408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:11.387408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:11.387408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:11.387408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:11.387408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:11.387408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:11.387408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:11.387408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:11.387408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:11.387408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:11.388749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:11.388749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:11.388749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:11.388749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:11.388749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 427/881] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:11.393790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9a187-694e-465c-a380-3aaaa801c41d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:11.485359 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e85e0-92e0-44f2-81cd-183070bf674f None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:11.485721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e85e0-92e0-44f2-81cd-183070bf674f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:11.485901 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e85e0-92e0-44f2-81cd-183070bf674f None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:11.487516 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 428/882] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:04:11.492396 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8c220-8fc5-404b-9f41-d11351e49834 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:11.492749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8c220-8fc5-404b-9f41-d11351e49834 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:11.493120 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8c220-8fc5-404b-9f41-d11351e49834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:11.500284 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9a187-694e-465c-a380-3aaaa801c41d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:11.511661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-78a8c220-8fc5-404b-9f41-d11351e49834 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:11.511661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-78a8c220-8fc5-404b-9f41-d11351e49834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:11.535431 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a7ce8-9c45-4596-9d9b-e586144e6b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:11.536129 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a7ce8-9c45-4596-9d9b-e586144e6b17 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:11.536407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a7ce8-9c45-4596-9d9b-e586144e6b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:11.536679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a7ce8-9c45-4596-9d9b-e586144e6b17 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:11.537404 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a7ce8-9c45-4596-9d9b-e586144e6b17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:11.537711 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a7ce8-9c45-4596-9d9b-e586144e6b17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:11.539102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a7ce8-9c45-4596-9d9b-e586144e6b17 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:11.556532 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 455/883] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:11.557790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a7ce8-9c45-4596-9d9b-e586144e6b17 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:11.570200 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:11.588998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 429/884] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:11.612149 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8c220-8fc5-404b-9f41-d11351e49834 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:11.668906 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a7ce8-9c45-4596-9d9b-e586144e6b17 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:11.688041 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:11.706621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7822a70-94f2-4122-b694-eac74ed1abbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:11.707453 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7822a70-94f2-4122-b694-eac74ed1abbc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:11.707771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7822a70-94f2-4122-b694-eac74ed1abbc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:11.708050 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7822a70-94f2-4122-b694-eac74ed1abbc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:11.715856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7822a70-94f2-4122-b694-eac74ed1abbc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:11.717384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7822a70-94f2-4122-b694-eac74ed1abbc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:11.736685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f7822a70-94f2-4122-b694-eac74ed1abbc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:04:11.736685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:11.736685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:11.736685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:11.736685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:11.736685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:11.736685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:11.736685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:11.736685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:11.736685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:11.736685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:11.736685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:11.736685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:04:11.736685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:04:11.736685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:04:11.736685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:04:11.737651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:11.737651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:11.737651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:04:11.737651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:11.737651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:04:11.737651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:11.737651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:11.737651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:11.737651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:11.737651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:11.737651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:11.737651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:11.737651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:11.737651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:11.737651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:11.737651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:11.737651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:11.737651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:11.738567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:11.738567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:11.738567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:04:11.738567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:04:11.738567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:04:11.738567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:04:11.738567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:04:11.738567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:04:11.738567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:04:11.738567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:11.739163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8c220-8fc5-404b-9f41-d11351e49834 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:11.739243 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 456/885] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Jun 27 21:04:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:11.743807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7822a70-94f2-4122-b694-eac74ed1abbc None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:11.783331 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ef425-5a91-4dc7-9ccf-f440c6df4a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:11.784473 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ef425-5a91-4dc7-9ccf-f440c6df4a5e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:11.784767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ef425-5a91-4dc7-9ccf-f440c6df4a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:11.785053 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ef425-5a91-4dc7-9ccf-f440c6df4a5e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:11.785735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ef425-5a91-4dc7-9ccf-f440c6df4a5e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:11.788484 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ef425-5a91-4dc7-9ccf-f440c6df4a5e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:11.809579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-9f7ef425-5a91-4dc7-9ccf-f440c6df4a5e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 21:04:11.809579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:11.809579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:11.809579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:11.809579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:11.809579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:11.809579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:11.809579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:11.809579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 21:04:11.809579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:11.809579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:11.809579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:11.809579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:11.809579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:11.809579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:11.809579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:11.810734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:11.810734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:11.810734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:11.810734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:11.810734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:11.810734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:11.810734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:11.810734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:11.810734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:11.810734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:11.810734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:11.810734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:11.810734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:11.810734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:11.810734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:11.810734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:11.812093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:11.812093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:11.812093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:11.812093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:11.812093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:11.812093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:11.812093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:11.812093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:11.812093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:11.812093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:11.812093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:11.812093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:11.812093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:11.812093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 21:04:11.812093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:11.813301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 430/886] 10.222.0.27 () {56 vars in 1132 bytes} [Mon Jun 27 21:04:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:11.819312 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ef425-5a91-4dc7-9ccf-f440c6df4a5e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:11.832041 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:11.872953 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7822a70-94f2-4122-b694-eac74ed1abbc None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:11.918127 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536606c-326f-4ae6-8a72-71292e562949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:11.919385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536606c-326f-4ae6-8a72-71292e562949 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:11.919994 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536606c-326f-4ae6-8a72-71292e562949 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:11.920422 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536606c-326f-4ae6-8a72-71292e562949 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:11.921727 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536606c-326f-4ae6-8a72-71292e562949 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:11.922199 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536606c-326f-4ae6-8a72-71292e562949 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:11.924392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536606c-326f-4ae6-8a72-71292e562949 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:11.935953 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ef425-5a91-4dc7-9ccf-f440c6df4a5e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:11.945646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 457/887] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Jun 27 21:04:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:11.946791 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee8da3-b7f5-4f23-850d-99142a5e9e19 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:11.947367 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee8da3-b7f5-4f23-850d-99142a5e9e19 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:11.947367 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee8da3-b7f5-4f23-850d-99142a5e9e19 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:11.948846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 458/888] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:11.951225 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee8da3-b7f5-4f23-850d-99142a5e9e19 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:11.964977 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:11.981034 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb274bc-5d73-4261-8b21-306bf9637fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:11.981722 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb274bc-5d73-4261-8b21-306bf9637fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:11.982005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb274bc-5d73-4261-8b21-306bf9637fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:11.982277 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb274bc-5d73-4261-8b21-306bf9637fd7 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:11.982992 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb274bc-5d73-4261-8b21-306bf9637fd7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:11.983298 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb274bc-5d73-4261-8b21-306bf9637fd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:11.984715 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb274bc-5d73-4261-8b21-306bf9637fd7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:11.989491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 431/889] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Jun 27 21:04:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:11.990680 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ae98a-45a0-4de5-9d2c-632743aa58a8 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:11.990972 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ae98a-45a0-4de5-9d2c-632743aa58a8 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:11.991232 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ae98a-45a0-4de5-9d2c-632743aa58a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.007354 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-9e5ae98a-45a0-4de5-9d2c-632743aa58a8 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:12.008592 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9e5ae98a-45a0-4de5-9d2c-632743aa58a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:12.057187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee8da3-b7f5-4f23-850d-99142a5e9e19 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:12.077401 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 432/890] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:12.079008 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89e88b-8d4a-4861-90ed-685a779fc43e None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.079299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89e88b-8d4a-4861-90ed-685a779fc43e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.079580 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89e88b-8d4a-4861-90ed-685a779fc43e None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.083759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 433/891] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:12.083759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89e88b-8d4a-4861-90ed-685a779fc43e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:12.086414 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:12.095363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b517b3-6c75-472a-9a1f-bd8dc106cc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:12.096062 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b517b3-6c75-472a-9a1f-bd8dc106cc5c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.096339 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b517b3-6c75-472a-9a1f-bd8dc106cc5c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.096610 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b517b3-6c75-472a-9a1f-bd8dc106cc5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.164920 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b517b3-6c75-472a-9a1f-bd8dc106cc5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:12.166382 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b517b3-6c75-472a-9a1f-bd8dc106cc5c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:12.185018 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89e88b-8d4a-4861-90ed-685a779fc43e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:12.201521 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:12.219927 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edba85-38ec-4013-9068-1a075dabba6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:12.220599 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edba85-38ec-4013-9068-1a075dabba6c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.220905 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edba85-38ec-4013-9068-1a075dabba6c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.221181 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edba85-38ec-4013-9068-1a075dabba6c None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.221869 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edba85-38ec-4013-9068-1a075dabba6c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:12.222179 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edba85-38ec-4013-9068-1a075dabba6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:12.223566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edba85-38ec-4013-9068-1a075dabba6c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:12.228298 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 434/892] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Jun 27 21:04:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:12.229438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb64511-6e59-459f-8233-bfa93deb59d2 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.229723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb64511-6e59-459f-8233-bfa93deb59d2 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.230003 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb64511-6e59-459f-8233-bfa93deb59d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.245874 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ceb64511-6e59-459f-8233-bfa93deb59d2 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:12.247074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ceb64511-6e59-459f-8233-bfa93deb59d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:12.271201 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 459/893] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:11 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:12.271201 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b517b3-6c75-472a-9a1f-bd8dc106cc5c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:12.302095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:12.314227 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 435/894] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:12 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:12.315068 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb64511-6e59-459f-8233-bfa93deb59d2 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:12.375440 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b517b3-6c75-472a-9a1f-bd8dc106cc5c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:12.409728 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe998f3-a3ec-40a9-a3b8-66d961a5e209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:12.410535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe998f3-a3ec-40a9-a3b8-66d961a5e209 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.410822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe998f3-a3ec-40a9-a3b8-66d961a5e209 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.411099 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe998f3-a3ec-40a9-a3b8-66d961a5e209 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.411790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe998f3-a3ec-40a9-a3b8-66d961a5e209 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:12.413190 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe998f3-a3ec-40a9-a3b8-66d961a5e209 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:12.414708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb64511-6e59-459f-8233-bfa93deb59d2 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:12.425166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-7fe998f3-a3ec-40a9-a3b8-66d961a5e209 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 21:04:12.425166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:12.425166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:12.425166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:12.425166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:12.425166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:12.425166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:12.425166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:12.425166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 21:04:12.425166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:12.425166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:12.425166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:12.425166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:12.425166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:12.425166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:12.425166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:12.425166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:12.426198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:12.426198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:12.426198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:12.426198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:12.426198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:12.426198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:12.426198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:12.426198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:12.426198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:12.426198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:12.426198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:12.426198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:12.426198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:12.426198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:12.426198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:12.426198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:12.427543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:12.427543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:12.427543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:12.427543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:12.427543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:12.427543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:12.427543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:12.427543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:12.427543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:12.427543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:12.427543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:12.427543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:12.427543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 21:04:12.427543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:12.428797 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:12.428881 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 460/895] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:04:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:12.428881 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe998f3-a3ec-40a9-a3b8-66d961a5e209 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:12.454047 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d1faf-5bd7-42e3-a50f-d14d75f8cbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:12.454716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d1faf-5bd7-42e3-a50f-d14d75f8cbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.455004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d1faf-5bd7-42e3-a50f-d14d75f8cbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.455258 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d1faf-5bd7-42e3-a50f-d14d75f8cbe7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.456095 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d1faf-5bd7-42e3-a50f-d14d75f8cbe7 None admin] RBAC: Adding query filter params (interface=public, service_id=b771c73df39346ea8922a09ff4bc28c9, region_id=RegionOne) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:12.456407 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d1faf-5bd7-42e3-a50f-d14d75f8cbe7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:12.457786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d1faf-5bd7-42e3-a50f-d14d75f8cbe7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:12.462721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 436/896] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Jun 27 21:04:12 2022] GET /identity/v3/endpoints?service_id=b771c73df39346ea8922a09ff4bc28c9&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:12.463494 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d1faf-5bd7-42e3-a50f-d14d75f8cbe7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:12.523694 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:12.534215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe998f3-a3ec-40a9-a3b8-66d961a5e209 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:12.567592 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d1faf-5bd7-42e3-a50f-d14d75f8cbe7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:12.568138 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64cb52-7f68-4110-9390-8902814a4ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:12.568938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64cb52-7f68-4110-9390-8902814a4ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.569221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64cb52-7f68-4110-9390-8902814a4ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.569493 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64cb52-7f68-4110-9390-8902814a4ee3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.580002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64cb52-7f68-4110-9390-8902814a4ee3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:12.581471 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64cb52-7f68-4110-9390-8902814a4ee3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:12.598445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-3d64cb52-7f68-4110-9390-8902814a4ee3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:12.598445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:12.598445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:12.598445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:12.598445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:12.598445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:12.598445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:12.598445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:12.598445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:12.598445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:12.598445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:12.598445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:12.598445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:12.598445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:12.598445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:12.599347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:12.599347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:12.599347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:12.599347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:12.599347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:12.599347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:12.599347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:12.599347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:12.599347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:12.599347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:12.599347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:12.599347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:12.599347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:12.599347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:12.599347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:12.599347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:12.599347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:12.600526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:12.600526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:12.600526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:12.600526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:12.600526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 461/897] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:12.600526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64cb52-7f68-4110-9390-8902814a4ee3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:12.607743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d50ce1-b707-4e18-aed5-5a9ea571a907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:12.608592 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d50ce1-b707-4e18-aed5-5a9ea571a907 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.608868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d50ce1-b707-4e18-aed5-5a9ea571a907 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.609138 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d50ce1-b707-4e18-aed5-5a9ea571a907 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.609790 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d50ce1-b707-4e18-aed5-5a9ea571a907 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:12.611215 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d50ce1-b707-4e18-aed5-5a9ea571a907 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:12.623885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-04d50ce1-b707-4e18-aed5-5a9ea571a907 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:04:12.623885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:12.623885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:12.623885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:12.623885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:12.623885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:12.623885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:12.623885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:12.623885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:04:12.623885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:12.623885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:12.623885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:12.623885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:12.623885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:12.623885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:12.623885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:12.623885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:12.625019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:12.625019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:12.625019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:12.625019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:12.625019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:12.625019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:12.625019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:12.625019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:12.625019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:12.625019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:12.625019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:12.625019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:12.625019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:12.625019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:12.625019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:12.625019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:12.625902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:12.625902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:12.625902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:12.625902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:12.625902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:12.625902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:12.625902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:12.625902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:12.625902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:12.625902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:12.625902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:12.625902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:12.625902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:04:12.625902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:12.625902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 437/898] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Jun 27 21:04:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:12.626724 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d50ce1-b707-4e18-aed5-5a9ea571a907 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:12.669235 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:12.714613 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64cb52-7f68-4110-9390-8902814a4ee3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:12.729866 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d50ce1-b707-4e18-aed5-5a9ea571a907 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:12.749179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9a7b7-d5b1-47c4-8b85-b5252e480b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:12.749861 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9a7b7-d5b1-47c4-8b85-b5252e480b45 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.750157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9a7b7-d5b1-47c4-8b85-b5252e480b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.750476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9a7b7-d5b1-47c4-8b85-b5252e480b45 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.751223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9a7b7-d5b1-47c4-8b85-b5252e480b45 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:12.751530 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9a7b7-d5b1-47c4-8b85-b5252e480b45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:12.752954 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9a7b7-d5b1-47c4-8b85-b5252e480b45 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:12.757662 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 462/899] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:04:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:12.758453 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9a7b7-d5b1-47c4-8b85-b5252e480b45 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:12.763750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3999e-f3f5-4091-9141-ac4ca1c18f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:12.764426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3999e-f3f5-4091-9141-ac4ca1c18f95 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.764710 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3999e-f3f5-4091-9141-ac4ca1c18f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.764985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3999e-f3f5-4091-9141-ac4ca1c18f95 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.765669 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3999e-f3f5-4091-9141-ac4ca1c18f95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:12.767111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3999e-f3f5-4091-9141-ac4ca1c18f95 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:12.771039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:12.772898 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 438/900] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Jun 27 21:04:12 2022] GET /identity/v3/services => generated 1268 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:04:12.773653 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3999e-f3f5-4091-9141-ac4ca1c18f95 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:12.878682 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9a7b7-d5b1-47c4-8b85-b5252e480b45 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:12.886870 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3999e-f3f5-4091-9141-ac4ca1c18f95 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:12.889182 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:12.917589 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076b6aa-09e1-4963-aa6e-64e0a7bd559e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:12.918278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076b6aa-09e1-4963-aa6e-64e0a7bd559e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.918559 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076b6aa-09e1-4963-aa6e-64e0a7bd559e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.918842 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076b6aa-09e1-4963-aa6e-64e0a7bd559e None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.919566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076b6aa-09e1-4963-aa6e-64e0a7bd559e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:12.919908 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076b6aa-09e1-4963-aa6e-64e0a7bd559e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:12.921316 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076b6aa-09e1-4963-aa6e-64e0a7bd559e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:12.935197 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86612e9f-5e3d-4f14-bc2b-34ca9ee26ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:12.935197 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86612e9f-5e3d-4f14-bc2b-34ca9ee26ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.935479 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86612e9f-5e3d-4f14-bc2b-34ca9ee26ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.935745 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86612e9f-5e3d-4f14-bc2b-34ca9ee26ec3 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.935965 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 463/901] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:12.937016 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86612e9f-5e3d-4f14-bc2b-34ca9ee26ec3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:12.937139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076b6aa-09e1-4963-aa6e-64e0a7bd559e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:12.937227 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86612e9f-5e3d-4f14-bc2b-34ca9ee26ec3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:12.938191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86612e9f-5e3d-4f14-bc2b-34ca9ee26ec3 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:12.948425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 439/902] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Jun 27 21:04:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:12.949603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86612e9f-5e3d-4f14-bc2b-34ca9ee26ec3 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:13.015710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:13.058919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076b6aa-09e1-4963-aa6e-64e0a7bd559e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:13.066177 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86612e9f-5e3d-4f14-bc2b-34ca9ee26ec3 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:13.096875 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0afcc-219d-45ea-b729-fbb3e7821ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:13.097627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0afcc-219d-45ea-b729-fbb3e7821ada None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:13.097909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0afcc-219d-45ea-b729-fbb3e7821ada None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:13.098183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0afcc-219d-45ea-b729-fbb3e7821ada None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:13.098894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0afcc-219d-45ea-b729-fbb3e7821ada None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:13.099260 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0afcc-219d-45ea-b729-fbb3e7821ada None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:13.100621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0afcc-219d-45ea-b729-fbb3e7821ada None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:13.102436 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42e889-aa95-44d3-90c3-50c767bb9c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:13.103568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42e889-aa95-44d3-90c3-50c767bb9c77 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:13.103885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42e889-aa95-44d3-90c3-50c767bb9c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:13.104166 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42e889-aa95-44d3-90c3-50c767bb9c77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:13.110434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 464/903] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:04:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:13.111181 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0afcc-219d-45ea-b729-fbb3e7821ada None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:13.111440 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42e889-aa95-44d3-90c3-50c767bb9c77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:13.112950 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42e889-aa95-44d3-90c3-50c767bb9c77 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:13.130932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-5a42e889-aa95-44d3-90c3-50c767bb9c77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:13.130932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:13.130932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:13.130932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:13.130932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:13.130932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:13.130932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:13.130932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:13.130932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:13.130932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:13.130932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:13.130932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:13.130932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:04:13.130932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:04:13.130932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:04:13.131873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:04:13.131873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:13.131873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:13.131873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:13.131873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:13.131873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:13.131873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:13.131873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:13.131873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:13.131873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:13.131873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:13.131873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:13.131873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:13.131873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:13.131873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:13.131873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:13.131873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:13.132763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:04:13.132763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:04:13.132763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:04:13.132763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:04:13.132763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:04:13.132763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:04:13.132763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:13.132763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:13.132763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 440/904] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:04:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:13.133243 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42e889-aa95-44d3-90c3-50c767bb9c77 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:13.163197 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:13.215658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0afcc-219d-45ea-b729-fbb3e7821ada None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:13.246605 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:13.248363 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42e889-aa95-44d3-90c3-50c767bb9c77 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:13.254257 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54428f0d-a173-4de6-b161-4fe52e09f285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:13.254938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54428f0d-a173-4de6-b161-4fe52e09f285 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:13.255215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54428f0d-a173-4de6-b161-4fe52e09f285 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:13.255493 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54428f0d-a173-4de6-b161-4fe52e09f285 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:13.256641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54428f0d-a173-4de6-b161-4fe52e09f285 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:13.257127 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54428f0d-a173-4de6-b161-4fe52e09f285 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:13.258889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54428f0d-a173-4de6-b161-4fe52e09f285 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:13.264315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 465/905] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Jun 27 21:04:13 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:13.265094 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54428f0d-a173-4de6-b161-4fe52e09f285 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:13.300204 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d581ed-d283-4795-8bc0-2bc635ecfc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:13.301316 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d581ed-d283-4795-8bc0-2bc635ecfc62 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:13.301897 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d581ed-d283-4795-8bc0-2bc635ecfc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:13.302514 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d581ed-d283-4795-8bc0-2bc635ecfc62 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:13.303408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d581ed-d283-4795-8bc0-2bc635ecfc62 None admin] RBAC: Adding query filter params (interface=public, service_id=a6fd861c45c74722973dd126a0a4a546, region_id=RegionOne) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:13.303813 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d581ed-d283-4795-8bc0-2bc635ecfc62 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:13.305433 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d581ed-d283-4795-8bc0-2bc635ecfc62 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:13.312878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 441/906] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Jun 27 21:04:13 2022] GET /identity/v3/endpoints?service_id=a6fd861c45c74722973dd126a0a4a546&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:13.320667 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d581ed-d283-4795-8bc0-2bc635ecfc62 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:13.391662 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54428f0d-a173-4de6-b161-4fe52e09f285 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:13.429224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:13.430826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d496330-6222-413e-b348-0f13443cfde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:13.431635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d496330-6222-413e-b348-0f13443cfde7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:13.432008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d496330-6222-413e-b348-0f13443cfde7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:13.432408 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d496330-6222-413e-b348-0f13443cfde7 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:13.433359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d496330-6222-413e-b348-0f13443cfde7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:13.433770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d496330-6222-413e-b348-0f13443cfde7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:13.435595 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d496330-6222-413e-b348-0f13443cfde7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:13.443083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d581ed-d283-4795-8bc0-2bc635ecfc62 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:13.448045 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 466/907] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:04:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:13.454395 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d496330-6222-413e-b348-0f13443cfde7 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:13.493374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac683f58-23e5-4e11-9ff5-823971363950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:13.494229 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac683f58-23e5-4e11-9ff5-823971363950 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:13.494521 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac683f58-23e5-4e11-9ff5-823971363950 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:13.494801 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac683f58-23e5-4e11-9ff5-823971363950 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:13.495501 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac683f58-23e5-4e11-9ff5-823971363950 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:13.497203 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac683f58-23e5-4e11-9ff5-823971363950 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:13.511646 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 442/908] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Jun 27 21:04:13 2022] GET /identity/v3/services => generated 1268 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:04:13.526884 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:13.585387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d496330-6222-413e-b348-0f13443cfde7 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:13.623892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a9b0e-1498-44f0-97eb-83fb2a2688af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:13.624580 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a9b0e-1498-44f0-97eb-83fb2a2688af None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:13.624867 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a9b0e-1498-44f0-97eb-83fb2a2688af None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:13.625147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a9b0e-1498-44f0-97eb-83fb2a2688af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:13.626091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a9b0e-1498-44f0-97eb-83fb2a2688af None admin] RBAC: Adding query filter params (role.id=6134821b0f644a8ea7655b40f2ab3cc3, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=e802da0b078e457da466a9a7b0f15085) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:13.626410 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a9b0e-1498-44f0-97eb-83fb2a2688af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:13.627917 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a9b0e-1498-44f0-97eb-83fb2a2688af None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:13.635143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 467/909] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:04:13 2022] GET /identity/v3/role_assignments?user.id=e802da0b078e457da466a9a7b0f15085&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=6134821b0f644a8ea7655b40f2ab3cc3 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:14.807251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaaf32-6eea-40ac-8320-4a0ce02bc20b None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:14.807807 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaaf32-6eea-40ac-8320-4a0ce02bc20b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:14.807807 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaaf32-6eea-40ac-8320-4a0ce02bc20b None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:14.809464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 443/910] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:14.814698 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a7409-a1bf-4424-9b15-3d29a921225d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:14.815099 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a7409-a1bf-4424-9b15-3d29a921225d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:14.815265 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a7409-a1bf-4424-9b15-3d29a921225d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:14.832415 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7a8a7409-a1bf-4424-9b15-3d29a921225d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:14.833634 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7a8a7409-a1bf-4424-9b15-3d29a921225d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:14.907984 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 468/911] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:14.925920 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaaf32-6eea-40ac-8320-4a0ce02bc20b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:15.032859 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaaf32-6eea-40ac-8320-4a0ce02bc20b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:15.068977 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c01d4c-95ad-4f65-91bc-94f88ccb10c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:15.069717 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c01d4c-95ad-4f65-91bc-94f88ccb10c2 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:15.069993 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c01d4c-95ad-4f65-91bc-94f88ccb10c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:15.070262 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c01d4c-95ad-4f65-91bc-94f88ccb10c2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:15.070920 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c01d4c-95ad-4f65-91bc-94f88ccb10c2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:15.072339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c01d4c-95ad-4f65-91bc-94f88ccb10c2 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:15.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-09c01d4c-95ad-4f65-91bc-94f88ccb10c2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 21:04:15.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:15.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:15.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:15.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:15.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:15.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:15.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:15.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 21:04:15.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:15.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:15.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:15.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:15.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:15.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:15.085137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:15.086122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:15.086122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:15.086122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:15.086122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:15.086122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:15.086122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:15.086122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:15.086122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:15.086122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:15.086122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:15.086122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:15.086122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:15.086122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:15.086122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:15.086122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:15.086122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:15.086957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:15.086957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:15.086957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:15.086957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:15.086957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:15.086957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:15.086957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:15.086957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:15.086957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:15.086957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:15.086957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:15.086957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:15.086957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:15.086957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 21:04:15.086957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:15.091347 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 444/912] 10.222.0.27 () {56 vars in 1132 bytes} [Mon Jun 27 21:04:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:15.092799 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a7409-a1bf-4424-9b15-3d29a921225d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:15.142788 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:15.197127 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a7409-a1bf-4424-9b15-3d29a921225d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:15.232710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4345c97-a7cc-4464-9f18-7bd6d85d3f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:15.233644 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4345c97-a7cc-4464-9f18-7bd6d85d3f29 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:15.234073 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4345c97-a7cc-4464-9f18-7bd6d85d3f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:15.234524 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4345c97-a7cc-4464-9f18-7bd6d85d3f29 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:15.235436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4345c97-a7cc-4464-9f18-7bd6d85d3f29 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:15.235914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4345c97-a7cc-4464-9f18-7bd6d85d3f29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:15.237447 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4345c97-a7cc-4464-9f18-7bd6d85d3f29 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:15.242871 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 469/913] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Jun 27 21:04:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:15.251723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c01d4c-95ad-4f65-91bc-94f88ccb10c2 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:15.271953 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:15.355384 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c01d4c-95ad-4f65-91bc-94f88ccb10c2 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:15.389241 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848c6ea-3ff2-48d9-a1e6-0a0dffc209e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:15.389914 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848c6ea-3ff2-48d9-a1e6-0a0dffc209e6 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:15.390192 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848c6ea-3ff2-48d9-a1e6-0a0dffc209e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:15.390464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848c6ea-3ff2-48d9-a1e6-0a0dffc209e6 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:15.391260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848c6ea-3ff2-48d9-a1e6-0a0dffc209e6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:15.391566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848c6ea-3ff2-48d9-a1e6-0a0dffc209e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:15.393021 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848c6ea-3ff2-48d9-a1e6-0a0dffc209e6 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:15.402677 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 445/914] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Jun 27 21:04:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:15.404071 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4345c97-a7cc-4464-9f18-7bd6d85d3f29 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:15.473641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916c4c7-b679-468a-b1d8-369aeb126126 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:15.474005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916c4c7-b679-468a-b1d8-369aeb126126 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:15.474190 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916c4c7-b679-468a-b1d8-369aeb126126 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:15.479376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 446/915] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:04:15.481063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43552f92-1d5f-4205-8ab8-bb9269d475c9 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:15.481351 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43552f92-1d5f-4205-8ab8-bb9269d475c9 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:15.481632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43552f92-1d5f-4205-8ab8-bb9269d475c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:15.498595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-43552f92-1d5f-4205-8ab8-bb9269d475c9 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:15.499878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-43552f92-1d5f-4205-8ab8-bb9269d475c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:15.510775 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4345c97-a7cc-4464-9f18-7bd6d85d3f29 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:15.527955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:15.552360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5df04-d1a3-4257-8d37-711109dd7b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:15.553402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5df04-d1a3-4257-8d37-711109dd7b24 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:15.553887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5df04-d1a3-4257-8d37-711109dd7b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:15.554399 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5df04-d1a3-4257-8d37-711109dd7b24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:15.555669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b5df04-d1a3-4257-8d37-711109dd7b24 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:15.557261 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b5df04-d1a3-4257-8d37-711109dd7b24 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:15.576059 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 447/916] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:15 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:15.578969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 470/917] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Jun 27 21:04:15 2022] POST /identity/v3/endpoints => generated 347 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:15.583650 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0d55f-1d41-43f3-86e9-6b20af2c5c0f None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:15.583650 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0d55f-1d41-43f3-86e9-6b20af2c5c0f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:15.583650 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0d55f-1d41-43f3-86e9-6b20af2c5c0f None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:15.583650 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 448/918] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:04:15.594030 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f662f-2a35-40c0-8020-ee3dafd50b03 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:15.594693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f662f-2a35-40c0-8020-ee3dafd50b03 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:15.595111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f662f-2a35-40c0-8020-ee3dafd50b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:15.597951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0d55f-1d41-43f3-86e9-6b20af2c5c0f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:15.618822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:15.623308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6d6f662f-2a35-40c0-8020-ee3dafd50b03 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:15.624821 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6d6f662f-2a35-40c0-8020-ee3dafd50b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:15.712957 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 471/919] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:15 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:15.729087 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdeb192-f41d-4f88-9354-960d902f2865 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:15.729087 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdeb192-f41d-4f88-9354-960d902f2865 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:15.729087 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdeb192-f41d-4f88-9354-960d902f2865 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:15.729087 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 472/920] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:15 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:15.730056 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be081c-387f-4181-9f94-12822cb5f51e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:15.730339 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be081c-387f-4181-9f94-12822cb5f51e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:15.730613 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be081c-387f-4181-9f94-12822cb5f51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:15.744541 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0d55f-1d41-43f3-86e9-6b20af2c5c0f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:15.749400 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a9be081c-387f-4181-9f94-12822cb5f51e None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:15.750630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a9be081c-387f-4181-9f94-12822cb5f51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:15.781034 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087aabc9-880b-4de5-b002-df1904dc6279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:15.781731 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087aabc9-880b-4de5-b002-df1904dc6279 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:15.782028 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087aabc9-880b-4de5-b002-df1904dc6279 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:15.782301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087aabc9-880b-4de5-b002-df1904dc6279 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:15.783374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087aabc9-880b-4de5-b002-df1904dc6279 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:15.784789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087aabc9-880b-4de5-b002-df1904dc6279 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:15.800279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 449/921] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Jun 27 21:04:15 2022] POST /identity/v3/services => generated 241 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:15.801104 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087aabc9-880b-4de5-b002-df1904dc6279 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:15.846412 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 473/922] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:15.870092 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be081c-387f-4181-9f94-12822cb5f51e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:15.903886 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:15.904461 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:15.954136 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087aabc9-880b-4de5-b002-df1904dc6279 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:16.008859 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a9d38-53d8-45c1-9f99-c41f93863e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:16.009692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a9d38-53d8-45c1-9f99-c41f93863e2c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:16.009973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a9d38-53d8-45c1-9f99-c41f93863e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:16.010246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a9d38-53d8-45c1-9f99-c41f93863e2c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:16.011038 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a9d38-53d8-45c1-9f99-c41f93863e2c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:16.012621 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a9d38-53d8-45c1-9f99-c41f93863e2c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:16.020982 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be081c-387f-4181-9f94-12822cb5f51e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:16.025707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-df8a9d38-53d8-45c1-9f99-c41f93863e2c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 21:04:16.025707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:16.025707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:16.025707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:16.025707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:16.025707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:16.025707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:16.025707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:16.025707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 21:04:16.025707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:16.025707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:16.025707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:16.025707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:16.025707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:16.025707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:16.025707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:16.025707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:16.026804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:16.026804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:16.026804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:16.026804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:16.026804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:16.026804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:16.026804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:16.026804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:16.026804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:16.026804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:16.026804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:16.026804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:16.026804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:16.026804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:16.026804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:16.026804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:16.027758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:16.027758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:16.027758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:16.027758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:16.027758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:16.027758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:16.027758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:16.027758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:16.027758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:16.027758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:16.027758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:16.027758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:16.027758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 21:04:16.027758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:16.027758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 450/923] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:04:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:16.035153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a9d38-53d8-45c1-9f99-c41f93863e2c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:16.072584 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60dbb04-e979-45e2-88a9-3ebd9f6bbbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:16.073709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dbb04-e979-45e2-88a9-3ebd9f6bbbb6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:16.073991 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dbb04-e979-45e2-88a9-3ebd9f6bbbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:16.074262 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dbb04-e979-45e2-88a9-3ebd9f6bbbb6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:16.088234 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60dbb04-e979-45e2-88a9-3ebd9f6bbbb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:16.099820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-a60dbb04-e979-45e2-88a9-3ebd9f6bbbb6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:04:16.099820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:16.099820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:16.099820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:16.099820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:16.099820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:16.099820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:16.099820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:16.099820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:16.099820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:16.099820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:16.099820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:16.099820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:16.099820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:16.099820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:16.099820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:16.101008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:16.101008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:16.101008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:16.101008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:16.101008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:16.101008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:16.101008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:16.101008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:16.101008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:16.101008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:16.101008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:16.101008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:16.101008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:16.101008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:16.101008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:16.101008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:16.102080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:16.102080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:16.102080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:16.102080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:16.102080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:16.102080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:16.102080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:16.102080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:16.102080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:16.102080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:16.102080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:04:16.102080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:16.102080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 474/924] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Jun 27 21:04:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:16.111641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60dbb04-e979-45e2-88a9-3ebd9f6bbbb6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:16.150839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a9d38-53d8-45c1-9f99-c41f93863e2c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:16.187987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab7d3c-6d40-4d25-81dd-9105fd80a8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:16.187987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab7d3c-6d40-4d25-81dd-9105fd80a8ce None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:16.188419 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab7d3c-6d40-4d25-81dd-9105fd80a8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:16.188685 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab7d3c-6d40-4d25-81dd-9105fd80a8ce None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:16.189609 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab7d3c-6d40-4d25-81dd-9105fd80a8ce None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:16.190115 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab7d3c-6d40-4d25-81dd-9105fd80a8ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:16.192165 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab7d3c-6d40-4d25-81dd-9105fd80a8ce None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:16.194079 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:16.198741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 451/925] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:04:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:16.205000 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:16.206346 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab7d3c-6d40-4d25-81dd-9105fd80a8ce None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:16.233537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60dbb04-e979-45e2-88a9-3ebd9f6bbbb6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:16.270834 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7eb86a-2e28-4cf5-8d01-8cb293bc09be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:16.271821 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7eb86a-2e28-4cf5-8d01-8cb293bc09be None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:16.272220 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7eb86a-2e28-4cf5-8d01-8cb293bc09be None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:16.272813 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7eb86a-2e28-4cf5-8d01-8cb293bc09be None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:16.273639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7eb86a-2e28-4cf5-8d01-8cb293bc09be None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:16.274013 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7eb86a-2e28-4cf5-8d01-8cb293bc09be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:16.275480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7eb86a-2e28-4cf5-8d01-8cb293bc09be None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:16.290747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 475/926] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Jun 27 21:04:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:16.292030 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7eb86a-2e28-4cf5-8d01-8cb293bc09be None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:16.316845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab7d3c-6d40-4d25-81dd-9105fd80a8ce None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:16.359186 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e274d8d-7115-435d-96e9-7908a71066ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:16.360424 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e274d8d-7115-435d-96e9-7908a71066ba None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:16.360969 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e274d8d-7115-435d-96e9-7908a71066ba None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:16.361451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e274d8d-7115-435d-96e9-7908a71066ba None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:16.362379 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e274d8d-7115-435d-96e9-7908a71066ba None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:16.362896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e274d8d-7115-435d-96e9-7908a71066ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:16.364529 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e274d8d-7115-435d-96e9-7908a71066ba None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:16.375306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 452/927] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Jun 27 21:04:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:16.376611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e274d8d-7115-435d-96e9-7908a71066ba None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:16.414150 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7eb86a-2e28-4cf5-8d01-8cb293bc09be None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:16.445802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:16.450112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cdd8e-c313-433b-a50d-7f1b55ee368e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:16.450927 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cdd8e-c313-433b-a50d-7f1b55ee368e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:16.451211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cdd8e-c313-433b-a50d-7f1b55ee368e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:16.451589 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cdd8e-c313-433b-a50d-7f1b55ee368e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:16.462905 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cdd8e-c313-433b-a50d-7f1b55ee368e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:16.464400 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cdd8e-c313-433b-a50d-7f1b55ee368e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:16.467029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:16.483124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-a41cdd8e-c313-433b-a50d-7f1b55ee368e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:16.483124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:16.483124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:16.483124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:16.483124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:16.483124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:16.483124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:16.483124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:16.483124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:16.483124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:16.483124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:16.483124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:16.483124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:16.483124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:16.483124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:16.484349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:16.484349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:16.484349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:16.484349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:16.484349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:16.484349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:16.484349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:16.484349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:16.484349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:16.484349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:16.484349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:16.484349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:16.484349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:16.484349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:16.484349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:16.484349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:16.484349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:16.485448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:16.485448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:16.485448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:16.485448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:16.485448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 476/928] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:16.490253 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cdd8e-c313-433b-a50d-7f1b55ee368e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:16.495799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e274d8d-7115-435d-96e9-7908a71066ba None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:16.531466 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f209c1c-d118-469f-89db-5eeb088d5679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:16.532940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f209c1c-d118-469f-89db-5eeb088d5679 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:16.533430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f209c1c-d118-469f-89db-5eeb088d5679 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:16.533909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f209c1c-d118-469f-89db-5eeb088d5679 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:16.534880 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f209c1c-d118-469f-89db-5eeb088d5679 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:16.536592 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f209c1c-d118-469f-89db-5eeb088d5679 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:16.571650 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 453/929] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Jun 27 21:04:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:16.607508 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cdd8e-c313-433b-a50d-7f1b55ee368e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:16.657500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c487b-80db-40bb-882c-86b9cd689681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:16.658676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c487b-80db-40bb-882c-86b9cd689681 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:16.659027 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c487b-80db-40bb-882c-86b9cd689681 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:16.659368 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c487b-80db-40bb-882c-86b9cd689681 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:16.660289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c487b-80db-40bb-882c-86b9cd689681 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:16.660717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c487b-80db-40bb-882c-86b9cd689681 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:16.662285 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c487b-80db-40bb-882c-86b9cd689681 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:16.676605 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 477/930] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:16.682580 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f209c1c-d118-469f-89db-5eeb088d5679 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:16.807833 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f209c1c-d118-469f-89db-5eeb088d5679 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:16.829532 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:16.846291 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae23a5c-7583-4bf3-b520-a06ca6d8636c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:16.847203 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae23a5c-7583-4bf3-b520-a06ca6d8636c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:16.847551 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae23a5c-7583-4bf3-b520-a06ca6d8636c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:16.847918 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae23a5c-7583-4bf3-b520-a06ca6d8636c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:16.856137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae23a5c-7583-4bf3-b520-a06ca6d8636c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:16.857761 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae23a5c-7583-4bf3-b520-a06ca6d8636c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:16.872691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-fae23a5c-7583-4bf3-b520-a06ca6d8636c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:04:16.872691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:16.872691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:16.872691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:16.872691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:16.872691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:16.872691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:16.872691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:16.872691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:16.872691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:16.872691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:16.872691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:16.872691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:04:16.872691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:04:16.872691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:04:16.872691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:04:16.873823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:16.873823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:16.873823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:04:16.873823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:16.873823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:04:16.873823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:16.873823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:16.873823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:16.873823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:16.873823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:16.873823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:16.873823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:16.873823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:16.873823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:16.873823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:16.873823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:16.873823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:16.873823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:16.874991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:16.874991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:16.874991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:04:16.874991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:04:16.874991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:04:16.874991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:04:16.874991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:04:16.874991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:04:16.874991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:04:16.874991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:16.876108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 454/931] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Jun 27 21:04:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:16.881751 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c487b-80db-40bb-882c-86b9cd689681 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:16.937725 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:16.988821 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c487b-80db-40bb-882c-86b9cd689681 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:17.029679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786179b6-8076-4ad8-8302-d600dc98c1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:17.030361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786179b6-8076-4ad8-8302-d600dc98c1b4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:17.030642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786179b6-8076-4ad8-8302-d600dc98c1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:17.030958 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786179b6-8076-4ad8-8302-d600dc98c1b4 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:17.031903 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786179b6-8076-4ad8-8302-d600dc98c1b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:17.032220 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786179b6-8076-4ad8-8302-d600dc98c1b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:17.034013 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786179b6-8076-4ad8-8302-d600dc98c1b4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:17.052106 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 478/932] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Jun 27 21:04:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:17.057468 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae23a5c-7583-4bf3-b520-a06ca6d8636c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:17.168881 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae23a5c-7583-4bf3-b520-a06ca6d8636c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:17.205719 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146b0f8-7802-4f6b-b643-2e0856ebf053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:17.206763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146b0f8-7802-4f6b-b643-2e0856ebf053 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:17.207189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146b0f8-7802-4f6b-b643-2e0856ebf053 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:17.207651 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146b0f8-7802-4f6b-b643-2e0856ebf053 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:17.218895 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146b0f8-7802-4f6b-b643-2e0856ebf053 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:17.220590 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146b0f8-7802-4f6b-b643-2e0856ebf053 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:17.241360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-a146b0f8-7802-4f6b-b643-2e0856ebf053 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:17.241360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:17.241360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:17.241360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:17.241360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:17.241360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:17.241360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:17.241360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:17.241360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:17.241360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:17.241360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:17.241360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:17.241360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:17.241360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:17.241360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:17.242549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:17.242549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:17.242549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:17.242549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:17.242549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:17.242549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:17.242549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:17.242549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:17.242549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:17.242549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:17.242549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:17.242549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:17.242549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:17.242549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:17.242549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:17.242549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:17.242549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:17.243647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:17.243647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:17.243647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:17.243647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:17.245540 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 455/933] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:17.251018 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786179b6-8076-4ad8-8302-d600dc98c1b4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:17.352159 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786179b6-8076-4ad8-8302-d600dc98c1b4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:17.363673 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:17.388000 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de242b-fbe4-4e6b-8b63-77106a1610fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:17.388677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de242b-fbe4-4e6b-8b63-77106a1610fc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:17.388958 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de242b-fbe4-4e6b-8b63-77106a1610fc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:17.389229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de242b-fbe4-4e6b-8b63-77106a1610fc None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:17.389948 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de242b-fbe4-4e6b-8b63-77106a1610fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:17.390251 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de242b-fbe4-4e6b-8b63-77106a1610fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:17.391674 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de242b-fbe4-4e6b-8b63-77106a1610fc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:17.403747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 479/934] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:17.409421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146b0f8-7802-4f6b-b643-2e0856ebf053 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:17.442553 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:17.519889 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146b0f8-7802-4f6b-b643-2e0856ebf053 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:17.555501 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b0266-2d66-46b9-98fc-8ce0f6fe21c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:17.556348 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b0266-2d66-46b9-98fc-8ce0f6fe21c8 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:17.556634 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b0266-2d66-46b9-98fc-8ce0f6fe21c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:17.556911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b0266-2d66-46b9-98fc-8ce0f6fe21c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:17.564045 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b0266-2d66-46b9-98fc-8ce0f6fe21c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:17.565515 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b0266-2d66-46b9-98fc-8ce0f6fe21c8 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:17.580110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-3d7b0266-2d66-46b9-98fc-8ce0f6fe21c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:17.580110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:17.580110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:17.580110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:17.580110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:17.580110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:17.580110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:17.580110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:17.580110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:17.580110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:17.580110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:17.580110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:17.580110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:04:17.580110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:04:17.580110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:04:17.580990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:04:17.580990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:17.580990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:17.580990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:17.580990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:17.580990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:17.580990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:17.580990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:17.580990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:17.580990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:17.580990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:17.580990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:17.580990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:17.580990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:17.580990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:17.580990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:17.580990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:17.581876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:04:17.581876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:04:17.581876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:04:17.581876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:04:17.581876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:04:17.581876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:04:17.581876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:17.581876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:17.581876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 456/935] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:04:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:17.588839 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de242b-fbe4-4e6b-8b63-77106a1610fc None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:17.664563 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bd32a-915f-46fd-80cd-04f9c04c5324 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:17.664896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bd32a-915f-46fd-80cd-04f9c04c5324 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:17.665162 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bd32a-915f-46fd-80cd-04f9c04c5324 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:17.666793 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 457/936] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:04:17.671756 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06801f0d-8373-43e0-a7a3-f7a4c496a370 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:17.672048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06801f0d-8373-43e0-a7a3-f7a4c496a370 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:17.672412 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06801f0d-8373-43e0-a7a3-f7a4c496a370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:17.690523 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-06801f0d-8373-43e0-a7a3-f7a4c496a370 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:17.691772 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-06801f0d-8373-43e0-a7a3-f7a4c496a370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:17.693754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de242b-fbe4-4e6b-8b63-77106a1610fc None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:17.727689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d07928-77fc-48c4-bbda-5745f68c9510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:17.728607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d07928-77fc-48c4-bbda-5745f68c9510 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:17.729064 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d07928-77fc-48c4-bbda-5745f68c9510 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:17.729475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d07928-77fc-48c4-bbda-5745f68c9510 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:17.730395 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d07928-77fc-48c4-bbda-5745f68c9510 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:17.730850 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d07928-77fc-48c4-bbda-5745f68c9510 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:17.732539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d07928-77fc-48c4-bbda-5745f68c9510 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:17.745751 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 480/937] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:04:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:17.753919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d07928-77fc-48c4-bbda-5745f68c9510 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:17.771785 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 458/938] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:17.783978 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:17.786709 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06801f0d-8373-43e0-a7a3-f7a4c496a370 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:17.812117 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:17.873012 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d07928-77fc-48c4-bbda-5745f68c9510 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:17.905511 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06801f0d-8373-43e0-a7a3-f7a4c496a370 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:17.909185 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b46ca-8ec2-4a73-a40d-178a2dd36351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:17.910185 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b46ca-8ec2-4a73-a40d-178a2dd36351 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:17.910536 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b46ca-8ec2-4a73-a40d-178a2dd36351 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:17.910876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b46ca-8ec2-4a73-a40d-178a2dd36351 None admin] PATH_INFO: `/v3/projects/dfc29b021d1646819137c89e93f33601/users/e802da0b078e457da466a9a7b0f15085/roles/6134821b0f644a8ea7655b40f2ab3cc3` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:17.913190 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b46ca-8ec2-4a73-a40d-178a2dd36351 None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc29b021d1646819137c89e93f33601, user_id=e802da0b078e457da466a9a7b0f15085, role_id=6134821b0f644a8ea7655b40f2ab3cc3)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:17.914707 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b46ca-8ec2-4a73-a40d-178a2dd36351 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:17.931649 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 481/939] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:04:17 2022] PUT /identity/v3/projects/dfc29b021d1646819137c89e93f33601/users/e802da0b078e457da466a9a7b0f15085/roles/6134821b0f644a8ea7655b40f2ab3cc3 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:04:17.952248 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503c57f-cbbf-4409-a6e5-2862b8d04e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:17.953492 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503c57f-cbbf-4409-a6e5-2862b8d04e5b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:17.954062 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503c57f-cbbf-4409-a6e5-2862b8d04e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:17.954508 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503c57f-cbbf-4409-a6e5-2862b8d04e5b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:17.955373 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503c57f-cbbf-4409-a6e5-2862b8d04e5b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:17.957123 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503c57f-cbbf-4409-a6e5-2862b8d04e5b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:17.970845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-1503c57f-cbbf-4409-a6e5-2862b8d04e5b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 21:04:17.970845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:17.970845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:17.970845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:17.970845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:17.970845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:17.970845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:17.970845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:17.970845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 21:04:17.970845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:17.970845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:17.970845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:17.970845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:17.970845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:17.970845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:17.970845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:17.970845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:17.972032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:17.972032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:17.972032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:17.972032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:17.972032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:17.972032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:17.972032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:17.972032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:17.972032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:17.972032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:17.972032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:17.972032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:17.972032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:17.972032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:17.972032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:17.972032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:17.972896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:17.972896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:17.972896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:17.972896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:17.972896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:17.972896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:17.972896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:17.972896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:17.972896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:17.972896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:17.972896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:17.972896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:17.972896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 21:04:17.972896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:17.973628 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 459/940] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:04:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:17.985459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b46ca-8ec2-4a73-a40d-178a2dd36351 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:18.001963 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbe811-21a9-494e-92e3-aabbfb4e420b None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:18.002386 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbe811-21a9-494e-92e3-aabbfb4e420b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:18.002756 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbe811-21a9-494e-92e3-aabbfb4e420b None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:18.005364 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 460/941] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:04:18.011092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512c40a-3141-4f75-8fe8-6288cd1569ed None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:18.011528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512c40a-3141-4f75-8fe8-6288cd1569ed None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:18.011840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512c40a-3141-4f75-8fe8-6288cd1569ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:18.030210 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-7512c40a-3141-4f75-8fe8-6288cd1569ed None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:18.031472 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7512c40a-3141-4f75-8fe8-6288cd1569ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:18.098890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:18.111122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 461/942] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:18.111686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b46ca-8ec2-4a73-a40d-178a2dd36351 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:18.133972 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512c40a-3141-4f75-8fe8-6288cd1569ed None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:18.150084 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f39f21-4585-4785-9c2d-b2a19d48f836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:18.151081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f39f21-4585-4785-9c2d-b2a19d48f836 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:18.151567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f39f21-4585-4785-9c2d-b2a19d48f836 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:18.152103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f39f21-4585-4785-9c2d-b2a19d48f836 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:18.153018 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f39f21-4585-4785-9c2d-b2a19d48f836 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:18.153567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f39f21-4585-4785-9c2d-b2a19d48f836 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:18.155211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f39f21-4585-4785-9c2d-b2a19d48f836 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:18.161455 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 482/943] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Jun 27 21:04:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:18.164015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:18.167290 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f39f21-4585-4785-9c2d-b2a19d48f836 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:18.264295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512c40a-3141-4f75-8fe8-6288cd1569ed None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:18.285354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f39f21-4585-4785-9c2d-b2a19d48f836 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:18.300758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6bfa02-4a38-4cf1-a5c7-7ccad423526c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:18.302159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bfa02-4a38-4cf1-a5c7-7ccad423526c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:18.302445 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bfa02-4a38-4cf1-a5c7-7ccad423526c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:18.302723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bfa02-4a38-4cf1-a5c7-7ccad423526c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:18.303629 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6bfa02-4a38-4cf1-a5c7-7ccad423526c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:18.305778 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6bfa02-4a38-4cf1-a5c7-7ccad423526c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:18.321743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-cd6bfa02-4a38-4cf1-a5c7-7ccad423526c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:04:18.321743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:18.321743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:18.321743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:18.321743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:18.321743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:18.321743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:18.321743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:18.321743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:04:18.321743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:18.321743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:18.321743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:18.321743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:18.321743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:18.321743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:18.321743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:18.321743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:18.322803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:18.322803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:18.322803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:18.322803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:18.322803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:18.322803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:18.322803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:18.322803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:18.322803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:18.322803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:18.322803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:18.322803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:18.322803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:18.322803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:18.322803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:18.322803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:18.323652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:18.323652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:18.323652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:18.323652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:18.323652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:18.323652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:18.323652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:18.323652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:18.323652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:18.323652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:18.323652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:18.323652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:18.323652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:04:18.323652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:18.323652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 462/944] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Jun 27 21:04:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:18.329125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6bfa02-4a38-4cf1-a5c7-7ccad423526c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:18.330460 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7557e13-d7f1-49c5-ad09-1b69af9e606a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:18.331151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7557e13-d7f1-49c5-ad09-1b69af9e606a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:18.331434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7557e13-d7f1-49c5-ad09-1b69af9e606a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:18.332039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7557e13-d7f1-49c5-ad09-1b69af9e606a None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:18.333726 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7557e13-d7f1-49c5-ad09-1b69af9e606a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:18.334053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7557e13-d7f1-49c5-ad09-1b69af9e606a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:18.335467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7557e13-d7f1-49c5-ad09-1b69af9e606a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:18.341250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 483/945] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Jun 27 21:04:18 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:18.428608 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:18.451756 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6bfa02-4a38-4cf1-a5c7-7ccad423526c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:18.489439 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2d2216-cc8a-4e79-8903-91ea02512ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:18.490133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d2216-cc8a-4e79-8903-91ea02512ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:18.490425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d2216-cc8a-4e79-8903-91ea02512ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:18.490704 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d2216-cc8a-4e79-8903-91ea02512ef7 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:18.491432 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2d2216-cc8a-4e79-8903-91ea02512ef7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:18.491781 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2d2216-cc8a-4e79-8903-91ea02512ef7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:18.493217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2d2216-cc8a-4e79-8903-91ea02512ef7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:18.499094 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 463/946] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Jun 27 21:04:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:18.504959 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7557e13-d7f1-49c5-ad09-1b69af9e606a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:18.531353 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:18.621153 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7557e13-d7f1-49c5-ad09-1b69af9e606a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:18.657305 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdf8b7-8703-44ec-a6b1-386f355598d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:18.658638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdf8b7-8703-44ec-a6b1-386f355598d7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:18.659100 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdf8b7-8703-44ec-a6b1-386f355598d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:18.659550 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdf8b7-8703-44ec-a6b1-386f355598d7 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:18.660488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdf8b7-8703-44ec-a6b1-386f355598d7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:18.660989 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdf8b7-8703-44ec-a6b1-386f355598d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:18.662565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdf8b7-8703-44ec-a6b1-386f355598d7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:18.668575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 484/947] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Jun 27 21:04:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:18.674802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2d2216-cc8a-4e79-8903-91ea02512ef7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:18.758471 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20179511-4ddb-4d80-b94a-208e7ad8f4a3 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:18.758838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20179511-4ddb-4d80-b94a-208e7ad8f4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:18.759032 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20179511-4ddb-4d80-b94a-208e7ad8f4a3 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:18.761302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 485/948] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:04:18.766468 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc408ad-715c-47d2-aaa1-e40fc409ea87 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:18.766787 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc408ad-715c-47d2-aaa1-e40fc409ea87 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:18.767077 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc408ad-715c-47d2-aaa1-e40fc409ea87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:18.784386 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3bc408ad-715c-47d2-aaa1-e40fc409ea87 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:18.785609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3bc408ad-715c-47d2-aaa1-e40fc409ea87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:18.789821 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2d2216-cc8a-4e79-8903-91ea02512ef7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:18.824657 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ceb8db-ca21-4c2f-b18b-fb2efa4c72e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:18.825725 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ceb8db-ca21-4c2f-b18b-fb2efa4c72e7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:18.826202 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ceb8db-ca21-4c2f-b18b-fb2efa4c72e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:18.826679 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ceb8db-ca21-4c2f-b18b-fb2efa4c72e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:18.827684 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ceb8db-ca21-4c2f-b18b-fb2efa4c72e7 None admin] RBAC: Adding query filter params (interface=public, service_id=7f78daafca654921adf1ae5a10ca18c4, region_id=RegionOne) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:18.828191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ceb8db-ca21-4c2f-b18b-fb2efa4c72e7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:18.829848 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ceb8db-ca21-4c2f-b18b-fb2efa4c72e7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:18.835749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 464/949] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Jun 27 21:04:18 2022] GET /identity/v3/endpoints?service_id=7f78daafca654921adf1ae5a10ca18c4&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:18.841033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ceb8db-ca21-4c2f-b18b-fb2efa4c72e7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:18.860015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:18.866249 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 486/950] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:18.892164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc408ad-715c-47d2-aaa1-e40fc409ea87 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:18.911178 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:18.971058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ceb8db-ca21-4c2f-b18b-fb2efa4c72e7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:19.004053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc408ad-715c-47d2-aaa1-e40fc409ea87 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:19.016314 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af82c0f-1570-4939-9ea6-7d30f80ae428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:19.017479 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af82c0f-1570-4939-9ea6-7d30f80ae428 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:19.017835 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af82c0f-1570-4939-9ea6-7d30f80ae428 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:19.018175 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af82c0f-1570-4939-9ea6-7d30f80ae428 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:19.019201 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af82c0f-1570-4939-9ea6-7d30f80ae428 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:19.020959 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af82c0f-1570-4939-9ea6-7d30f80ae428 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:19.035636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 465/951] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Jun 27 21:04:18 2022] GET /identity/v3/services => generated 1497 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:04:19.046794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba0243-b3f4-4483-8df1-b80a2efa738c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:19.047561 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba0243-b3f4-4483-8df1-b80a2efa738c None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:19.047873 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba0243-b3f4-4483-8df1-b80a2efa738c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:19.048146 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba0243-b3f4-4483-8df1-b80a2efa738c None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:19.048878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba0243-b3f4-4483-8df1-b80a2efa738c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:19.050360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba0243-b3f4-4483-8df1-b80a2efa738c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:19.051892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: WARNING keystone.server.flask.common [None req-4dba0243-b3f4-4483-8df1-b80a2efa738c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 27 21:04:19.074710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.sql.core [None req-4dba0243-b3f4-4483-8df1-b80a2efa738c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 27 21:04:19.074710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 27 21:04:19.074710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [parameters: {'id': 'ba55f3a9928e470191a56325daad0fec', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 27 21:04:19.074710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80400) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 27 21:04:19.092789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4dba0243-b3f4-4483-8df1-b80a2efa738c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ba55f3a9928e470191a56325daad0fec.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ba55f3a9928e470191a56325daad0fec. Jun 27 21:04:19.092789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:19.092789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 21:04:19.092789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 21:04:19.092789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 21:04:19.092789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 21:04:19.092789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 21:04:19.092789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 21:04:19.092789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 21:04:19.092789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application conn.query(q) Jun 27 21:04:19.092789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 21:04:19.092789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 21:04:19.092789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 21:04:19.093686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application result.read() Jun 27 21:04:19.093686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 21:04:19.093686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 21:04:19.093686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 21:04:19.093686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 21:04:19.093686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 21:04:19.093686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 21:04:19.093686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 21:04:19.093686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 21:04:19.093686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 27 21:04:19.093686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:19.093686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 27 21:04:19.093686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:19.093686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:19.093686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 27 21:04:19.093686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 27 21:04:19.093686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 27 21:04:19.093686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 27 21:04:19.094783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 21:04:19.094783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application next(self.gen) Jun 27 21:04:19.094783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 27 21:04:19.094783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application yield resource Jun 27 21:04:19.094783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 21:04:19.094783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application next(self.gen) Jun 27 21:04:19.094783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 27 21:04:19.094783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self.session.rollback() Jun 27 21:04:19.094783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 27 21:04:19.094783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self.force_reraise() Jun 27 21:04:19.094783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 27 21:04:19.094783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise self.value Jun 27 21:04:19.094783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 27 21:04:19.094783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 27 21:04:19.094783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 27 21:04:19.094783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application session.commit() Jun 27 21:04:19.094783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 27 21:04:19.095709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 27 21:04:19.095709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 27 21:04:19.095709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._prepare_impl() Jun 27 21:04:19.095709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 27 21:04:19.095709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self.session.flush() Jun 27 21:04:19.095709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 27 21:04:19.095709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._flush(objects) Jun 27 21:04:19.095709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 27 21:04:19.095709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 27 21:04:19.095709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 27 21:04:19.095709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application compat.raise_( Jun 27 21:04:19.095709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 21:04:19.095709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception Jun 27 21:04:19.095709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 27 21:04:19.095709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application flush_context.execute() Jun 27 21:04:19.095709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 27 21:04:19.095709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rec.execute(self) Jun 27 21:04:19.096746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 27 21:04:19.096746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 27 21:04:19.096746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 27 21:04:19.096746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 27 21:04:19.096746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 27 21:04:19.096746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 27 21:04:19.096746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 27 21:04:19.096746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 27 21:04:19.096746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 27 21:04:19.096746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 27 21:04:19.096746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 27 21:04:19.096746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 27 21:04:19.096746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 27 21:04:19.096746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 27 21:04:19.096746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 27 21:04:19.097538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 27 21:04:19.097538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 21:04:19.097538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception Jun 27 21:04:19.097538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 21:04:19.097538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 21:04:19.097538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 21:04:19.097538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 21:04:19.097538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 21:04:19.097538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 21:04:19.097538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 21:04:19.097538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application conn.query(q) Jun 27 21:04:19.097538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 21:04:19.097538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 21:04:19.097538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 21:04:19.097538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application result.read() Jun 27 21:04:19.097538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 21:04:19.097538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 21:04:19.098380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 21:04:19.098380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 21:04:19.098380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 21:04:19.098380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 21:04:19.098380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 21:04:19.098380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 21:04:19.098380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 27 21:04:19.098380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 27 21:04:19.098380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [parameters: {'id': 'ba55f3a9928e470191a56325daad0fec', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 27 21:04:19.098380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 27 21:04:19.098380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:19.098380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:19.098380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:19.098380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:19.098380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 27 21:04:19.099168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 27 21:04:19.099168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 27 21:04:19.099168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 27 21:04:19.099168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 27 21:04:19.099168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:19.099168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:19.099168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:19.099168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:19.099168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:19.099168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:19.099168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:19.099168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:19.099168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:19.099168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:19.099168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:19.099168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:19.099168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:19.101974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:19.101974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 27 21:04:19.101974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 27 21:04:19.101974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:19.101974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:19.101974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 27 21:04:19.101974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 27 21:04:19.101974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ba55f3a9928e470191a56325daad0fec. Jun 27 21:04:19.101974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:19.101974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 487/952] 10.222.0.27 () {60 vars in 1140 bytes} [Mon Jun 27 21:04:18 2022] POST /identity/v3/projects => generated 262 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:19.103578 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af82c0f-1570-4939-9ea6-7d30f80ae428 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:19.184537 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:19.217011 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af82c0f-1570-4939-9ea6-7d30f80ae428 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:19.251959 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e813731-3216-42da-b7e3-40c204ffa16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:19.252767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e813731-3216-42da-b7e3-40c204ffa16e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:19.253043 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e813731-3216-42da-b7e3-40c204ffa16e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:19.253320 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e813731-3216-42da-b7e3-40c204ffa16e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:19.260886 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e813731-3216-42da-b7e3-40c204ffa16e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:19.262347 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e813731-3216-42da-b7e3-40c204ffa16e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:19.276619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-8e813731-3216-42da-b7e3-40c204ffa16e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:19.276619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:19.276619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:19.276619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:19.276619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:19.276619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:19.276619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:19.276619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:19.276619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:19.276619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:19.276619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:19.276619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:19.276619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:04:19.276619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:04:19.276619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:04:19.278647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:04:19.278647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:19.278647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:19.278647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:19.278647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:19.278647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:19.278647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:19.278647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:19.278647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:19.278647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:19.278647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:19.278647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:19.278647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:19.278647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:19.278647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:19.278647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:19.278647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:19.280984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:04:19.280984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:04:19.280984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:04:19.280984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:04:19.280984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:04:19.280984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:04:19.280984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:19.280984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:19.285147 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 466/953] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:04:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:19.285334 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba0243-b3f4-4483-8df1-b80a2efa738c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:19.353984 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:19.389404 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba0243-b3f4-4483-8df1-b80a2efa738c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:19.423761 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aecad-1114-4777-9b0d-daea78b51693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:19.424433 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aecad-1114-4777-9b0d-daea78b51693 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:19.424712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aecad-1114-4777-9b0d-daea78b51693 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:19.424986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aecad-1114-4777-9b0d-daea78b51693 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:19.425738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5aecad-1114-4777-9b0d-daea78b51693 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:19.426050 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5aecad-1114-4777-9b0d-daea78b51693 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:19.427707 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5aecad-1114-4777-9b0d-daea78b51693 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:19.455859 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 488/954] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Jun 27 21:04:19 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:20.011140 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866ee58-cd6c-4bd1-b95b-e3b3a2e3afbb None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:20.011459 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866ee58-cd6c-4bd1-b95b-e3b3a2e3afbb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:20.011735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866ee58-cd6c-4bd1-b95b-e3b3a2e3afbb None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:20.013450 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 467/955] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:20.018930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d0615-dff6-4c45-95d8-a5dd3fd1385d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:20.019434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d0615-dff6-4c45-95d8-a5dd3fd1385d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:20.019540 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d0615-dff6-4c45-95d8-a5dd3fd1385d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:20.037050 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a08d0615-dff6-4c45-95d8-a5dd3fd1385d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:20.038549 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a08d0615-dff6-4c45-95d8-a5dd3fd1385d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:20.110637 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 489/956] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:20.133166 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866ee58-cd6c-4bd1-b95b-e3b3a2e3afbb None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:20.242716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866ee58-cd6c-4bd1-b95b-e3b3a2e3afbb None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:20.276968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:20.283793 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21bb54-eea8-4390-a6e6-de12908a2767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:20.284813 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21bb54-eea8-4390-a6e6-de12908a2767 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:20.285244 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21bb54-eea8-4390-a6e6-de12908a2767 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:20.285694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21bb54-eea8-4390-a6e6-de12908a2767 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:20.296976 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f21bb54-eea8-4390-a6e6-de12908a2767 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:20.305742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-5f21bb54-eea8-4390-a6e6-de12908a2767 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:04:20.305742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:20.305742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:20.305742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:20.305742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:20.305742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:20.305742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:20.305742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:20.305742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:20.305742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:20.305742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:20.305742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:20.305742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:20.305742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:20.305742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:20.305742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:20.307020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:20.307020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:20.307020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:20.307020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:20.307020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:20.307020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:20.307020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:20.307020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:20.307020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:20.307020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:20.307020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:20.307020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:20.307020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:20.307020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:20.307020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:20.307020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:20.308125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:20.308125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:20.308125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:20.308125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:20.308125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:20.308125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:20.308125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:20.308125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:20.308125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:20.308125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:20.308125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:04:20.308125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:20.310602 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 468/957] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Jun 27 21:04:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:20.316336 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d0615-dff6-4c45-95d8-a5dd3fd1385d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:20.372316 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:20.420667 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d0615-dff6-4c45-95d8-a5dd3fd1385d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:20.453759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e4e65b-570c-47bf-b781-0e0ba0972920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:20.454435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4e65b-570c-47bf-b781-0e0ba0972920 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:20.454713 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4e65b-570c-47bf-b781-0e0ba0972920 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:20.454983 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4e65b-570c-47bf-b781-0e0ba0972920 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:20.455747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e4e65b-570c-47bf-b781-0e0ba0972920 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:20.456055 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e4e65b-570c-47bf-b781-0e0ba0972920 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:20.457433 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e4e65b-570c-47bf-b781-0e0ba0972920 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:20.465569 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f128e7-c72e-43af-8a23-7e05db59f55e None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:20.465864 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f128e7-c72e-43af-8a23-7e05db59f55e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:20.466143 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f128e7-c72e-43af-8a23-7e05db59f55e None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:20.467749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 469/959] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:20.467871 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 490/959] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Jun 27 21:04:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:20.472735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fac8fc-d14a-46b5-8c65-5b2eb5bf678f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:20.473072 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fac8fc-d14a-46b5-8c65-5b2eb5bf678f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:20.473355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fac8fc-d14a-46b5-8c65-5b2eb5bf678f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:20.476451 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e4e65b-570c-47bf-b781-0e0ba0972920 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:20.491320 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-47fac8fc-d14a-46b5-8c65-5b2eb5bf678f None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:20.493029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-47fac8fc-d14a-46b5-8c65-5b2eb5bf678f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:20.572880 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 470/960] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:20.583435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e4e65b-570c-47bf-b781-0e0ba0972920 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:20.596282 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fac8fc-d14a-46b5-8c65-5b2eb5bf678f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:20.607682 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:20.623963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9d732-0f59-4c70-a79e-e9f5516adb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:20.625151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9d732-0f59-4c70-a79e-e9f5516adb24 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:20.625634 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9d732-0f59-4c70-a79e-e9f5516adb24 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:20.626063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9d732-0f59-4c70-a79e-e9f5516adb24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:20.642145 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9d732-0f59-4c70-a79e-e9f5516adb24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:20.643970 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9d732-0f59-4c70-a79e-e9f5516adb24 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:20.663961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-73e9d732-0f59-4c70-a79e-e9f5516adb24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:20.663961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:20.663961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:20.663961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:20.663961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:20.663961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:20.663961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:20.663961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:20.663961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:20.663961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:20.663961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:20.663961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:20.663961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:20.663961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:20.663961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:20.664916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:20.664916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:20.664916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:20.664916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:20.664916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:20.664916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:20.664916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:20.664916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:20.664916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:20.664916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:20.664916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:20.664916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:20.664916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:20.664916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:20.664916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:20.664916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:20.664916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:20.665828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:20.665828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:20.665828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:20.665828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:20.665828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 491/961] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:20.671149 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9d732-0f59-4c70-a79e-e9f5516adb24 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:20.724087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fac8fc-d14a-46b5-8c65-5b2eb5bf678f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:20.756222 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:20.773104 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e6e90-2ba4-41f0-99ef-afe3bef8d032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:20.774646 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e6e90-2ba4-41f0-99ef-afe3bef8d032 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:20.775089 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e6e90-2ba4-41f0-99ef-afe3bef8d032 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:20.775518 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e6e90-2ba4-41f0-99ef-afe3bef8d032 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:20.777076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e6e90-2ba4-41f0-99ef-afe3bef8d032 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:20.779143 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e6e90-2ba4-41f0-99ef-afe3bef8d032 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:20.783556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9d732-0f59-4c70-a79e-e9f5516adb24 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:20.803238 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 471/962] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Jun 27 21:04:20 2022] POST /identity/v3/services => generated 242 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:04:20.839589 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3090866-a31a-43b0-baa3-0a7353405428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:20.840740 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3090866-a31a-43b0-baa3-0a7353405428 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:20.841257 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3090866-a31a-43b0-baa3-0a7353405428 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:20.841720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3090866-a31a-43b0-baa3-0a7353405428 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:20.842754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3090866-a31a-43b0-baa3-0a7353405428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:20.843317 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3090866-a31a-43b0-baa3-0a7353405428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:20.845393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3090866-a31a-43b0-baa3-0a7353405428 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:20.861406 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 492/963] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:20.867644 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e6e90-2ba4-41f0-99ef-afe3bef8d032 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:20.986187 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e6e90-2ba4-41f0-99ef-afe3bef8d032 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:21.027736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefef18-7068-4809-8fe9-560a611f566a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:21.028568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefef18-7068-4809-8fe9-560a611f566a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:21.028848 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefef18-7068-4809-8fe9-560a611f566a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:21.029120 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefef18-7068-4809-8fe9-560a611f566a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:21.037637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fefef18-7068-4809-8fe9-560a611f566a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:21.039129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fefef18-7068-4809-8fe9-560a611f566a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:21.054048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-7fefef18-7068-4809-8fe9-560a611f566a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:04:21.054048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:21.054048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:21.054048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:21.054048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:21.054048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:21.054048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:21.054048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:21.054048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:21.054048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:21.054048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:21.054048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:21.054048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:04:21.054048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:04:21.054048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:04:21.054048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:04:21.055002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:21.055002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:21.055002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:04:21.055002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:21.055002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:04:21.055002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:21.055002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:21.055002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:21.055002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:21.055002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:21.055002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:21.055002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:21.055002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:21.055002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:21.055002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:21.055002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:21.055002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:21.055002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:21.056007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:21.056007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:21.056007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:04:21.056007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:04:21.056007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:04:21.056007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:04:21.056007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:04:21.056007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:04:21.056007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:04:21.056007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:21.056007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 472/964] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Jun 27 21:04:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:21.061393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3090866-a31a-43b0-baa3-0a7353405428 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:21.091084 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:21.166194 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3090866-a31a-43b0-baa3-0a7353405428 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:21.201241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245dfc0-29b1-4fd7-8f32-a5afd7ad3e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:21.201931 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245dfc0-29b1-4fd7-8f32-a5afd7ad3e02 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:21.202206 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245dfc0-29b1-4fd7-8f32-a5afd7ad3e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:21.202478 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245dfc0-29b1-4fd7-8f32-a5afd7ad3e02 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:21.203311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245dfc0-29b1-4fd7-8f32-a5afd7ad3e02 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:21.203669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245dfc0-29b1-4fd7-8f32-a5afd7ad3e02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:21.205113 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245dfc0-29b1-4fd7-8f32-a5afd7ad3e02 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:21.223413 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 493/965] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Jun 27 21:04:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:21.229133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefef18-7068-4809-8fe9-560a611f566a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:21.231054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafbf5c-b9e7-4e46-9123-fe824c7d3aa7 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:21.231339 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafbf5c-b9e7-4e46-9123-fe824c7d3aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:21.231637 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafbf5c-b9e7-4e46-9123-fe824c7d3aa7 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:21.233292 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 494/966] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:21.238355 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd6a40-a2e7-4b18-b3e5-13a184cadca7 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:21.238647 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd6a40-a2e7-4b18-b3e5-13a184cadca7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:21.238931 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd6a40-a2e7-4b18-b3e5-13a184cadca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:21.256904 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-65bd6a40-a2e7-4b18-b3e5-13a184cadca7 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:21.258114 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-65bd6a40-a2e7-4b18-b3e5-13a184cadca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:21.277975 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:21.335389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 495/967] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:21.352609 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefef18-7068-4809-8fe9-560a611f566a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:21.359992 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd6a40-a2e7-4b18-b3e5-13a184cadca7 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:21.378086 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:21.388911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309f70c-3f02-4141-942e-e46887302dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:21.389635 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309f70c-3f02-4141-942e-e46887302dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:21.389924 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309f70c-3f02-4141-942e-e46887302dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:21.390207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309f70c-3f02-4141-942e-e46887302dc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:21.463077 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309f70c-3f02-4141-942e-e46887302dc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:21.464607 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309f70c-3f02-4141-942e-e46887302dc2 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:21.465902 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd6a40-a2e7-4b18-b3e5-13a184cadca7 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:21.501456 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2ae89-93c4-423e-8c0e-3a1b46a73621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:21.502521 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2ae89-93c4-423e-8c0e-3a1b46a73621 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:21.502962 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2ae89-93c4-423e-8c0e-3a1b46a73621 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:21.503385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2ae89-93c4-423e-8c0e-3a1b46a73621 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:21.504225 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d2ae89-93c4-423e-8c0e-3a1b46a73621 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:21.505749 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d2ae89-93c4-423e-8c0e-3a1b46a73621 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:21.508929 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:21.519435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-01d2ae89-93c4-423e-8c0e-3a1b46a73621 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:04:21.519435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:21.519435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:21.519435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:21.519435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:21.519435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:21.519435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:21.519435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:21.519435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:04:21.519435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:21.519435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:21.519435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:21.519435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:21.519435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:21.519435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:21.519435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:21.519435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:21.520914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:21.520914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:21.520914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:21.520914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:21.520914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:21.520914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:21.520914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:21.520914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:21.520914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:21.520914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:21.520914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:21.520914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:21.520914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:21.520914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:21.520914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:21.520914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:21.521890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:21.521890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:21.521890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:21.521890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:21.521890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:21.521890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:21.521890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:21.521890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:21.521890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:21.521890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:21.521890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:21.521890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:21.521890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:04:21.521890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:21.529028 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 496/968] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Jun 27 21:04:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:21.530045 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2ae89-93c4-423e-8c0e-3a1b46a73621 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:21.576266 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 473/969] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:21 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:21.576266 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19fc3b-792b-4a98-bb3a-0902a532a58b None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:21.576266 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19fc3b-792b-4a98-bb3a-0902a532a58b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:21.576266 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19fc3b-792b-4a98-bb3a-0902a532a58b None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:21.576266 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 474/970] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:21.577941 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19fc3b-792b-4a98-bb3a-0902a532a58b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:21.620138 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:21.640525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2ae89-93c4-423e-8c0e-3a1b46a73621 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:21.677916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8229af-35e7-4f87-ad14-aed6751b03c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:21.678690 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8229af-35e7-4f87-ad14-aed6751b03c0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:21.678970 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8229af-35e7-4f87-ad14-aed6751b03c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:21.679240 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8229af-35e7-4f87-ad14-aed6751b03c0 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:21.679985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8229af-35e7-4f87-ad14-aed6751b03c0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:21.680289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8229af-35e7-4f87-ad14-aed6751b03c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:21.681656 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8229af-35e7-4f87-ad14-aed6751b03c0 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:21.687032 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 497/971] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Jun 27 21:04:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:21.688321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557086b2-333e-4cd0-bf8d-23d0fb6659df None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:21.688592 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557086b2-333e-4cd0-bf8d-23d0fb6659df None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:21.688864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557086b2-333e-4cd0-bf8d-23d0fb6659df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:21.693446 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19fc3b-792b-4a98-bb3a-0902a532a58b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:21.706665 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-557086b2-333e-4cd0-bf8d-23d0fb6659df None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:21.707909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-557086b2-333e-4cd0-bf8d-23d0fb6659df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:21.730460 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac141f-cd4f-4e47-9d5f-930bfc3ef411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:21.731275 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac141f-cd4f-4e47-9d5f-930bfc3ef411 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:21.731555 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac141f-cd4f-4e47-9d5f-930bfc3ef411 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:21.731925 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac141f-cd4f-4e47-9d5f-930bfc3ef411 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:21.741906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac141f-cd4f-4e47-9d5f-930bfc3ef411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:21.743353 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac141f-cd4f-4e47-9d5f-930bfc3ef411 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:21.745331 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:21.761826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-f0ac141f-cd4f-4e47-9d5f-930bfc3ef411 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:21.761826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:21.761826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:21.761826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:21.761826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:21.761826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:21.761826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:21.761826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:21.761826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:21.761826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:21.761826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:21.761826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:21.761826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:21.761826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:21.761826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:21.762726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:21.762726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:21.762726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:21.762726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:21.762726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:21.762726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:21.762726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:21.762726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:21.762726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:21.762726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:21.762726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:21.762726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:21.762726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:21.762726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:21.762726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:21.762726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:21.762726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:21.763675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:21.763675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:21.763675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:21.763675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:21.763675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 475/972] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:21.768417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac141f-cd4f-4e47-9d5f-930bfc3ef411 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:21.783969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 498/973] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:21.785416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557086b2-333e-4cd0-bf8d-23d0fb6659df None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:21.865682 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:21.877382 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac141f-cd4f-4e47-9d5f-930bfc3ef411 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:21.892560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557086b2-333e-4cd0-bf8d-23d0fb6659df None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:21.919057 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96ece5-724d-4291-9cce-12d9a7c601d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:21.919760 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96ece5-724d-4291-9cce-12d9a7c601d8 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:21.920042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96ece5-724d-4291-9cce-12d9a7c601d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:21.920314 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96ece5-724d-4291-9cce-12d9a7c601d8 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:21.921022 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96ece5-724d-4291-9cce-12d9a7c601d8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:21.921412 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96ece5-724d-4291-9cce-12d9a7c601d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:21.922745 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96ece5-724d-4291-9cce-12d9a7c601d8 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:21.932466 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 476/974] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Jun 27 21:04:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:21.932466 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96ece5-724d-4291-9cce-12d9a7c601d8 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:21.939968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34582e29-ab3e-4cc4-b3af-fb0d20e73367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:21.940966 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34582e29-ab3e-4cc4-b3af-fb0d20e73367 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:21.941279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34582e29-ab3e-4cc4-b3af-fb0d20e73367 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:21.941818 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34582e29-ab3e-4cc4-b3af-fb0d20e73367 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:21.942546 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34582e29-ab3e-4cc4-b3af-fb0d20e73367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:21.942852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34582e29-ab3e-4cc4-b3af-fb0d20e73367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:21.946274 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34582e29-ab3e-4cc4-b3af-fb0d20e73367 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:21.963959 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 499/975] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:21.964714 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34582e29-ab3e-4cc4-b3af-fb0d20e73367 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:21.999950 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:22.046257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96ece5-724d-4291-9cce-12d9a7c601d8 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:22.080655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34582e29-ab3e-4cc4-b3af-fb0d20e73367 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:22.083232 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67709f1a-61f7-430e-b287-02bf1f73d65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:22.084082 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67709f1a-61f7-430e-b287-02bf1f73d65d None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:22.084360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67709f1a-61f7-430e-b287-02bf1f73d65d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:22.084634 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67709f1a-61f7-430e-b287-02bf1f73d65d None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:22.085339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67709f1a-61f7-430e-b287-02bf1f73d65d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:22.086792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67709f1a-61f7-430e-b287-02bf1f73d65d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:22.088136 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: WARNING keystone.server.flask.common [None req-67709f1a-61f7-430e-b287-02bf1f73d65d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 27 21:04:22.101533 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.sql.core [None req-67709f1a-61f7-430e-b287-02bf1f73d65d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 27 21:04:22.101533 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 27 21:04:22.101533 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [parameters: {'user_id': 'ee44b7b99ce14843b76dbc528e4eb077', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 27 21:04:22.101533 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80401) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 27 21:04:22.109749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-67709f1a-61f7-430e-b287-02bf1f73d65d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 27 21:04:22.109749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:22.109749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 21:04:22.109749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 21:04:22.109749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 21:04:22.109749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 21:04:22.109749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 21:04:22.109749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 21:04:22.109749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 21:04:22.109749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application conn.query(q) Jun 27 21:04:22.109749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 21:04:22.109749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 21:04:22.109749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 21:04:22.109749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application result.read() Jun 27 21:04:22.109749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 21:04:22.110773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 21:04:22.110773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 21:04:22.110773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 21:04:22.110773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 21:04:22.110773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 21:04:22.110773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 21:04:22.110773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 21:04:22.110773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 27 21:04:22.110773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:22.110773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 27 21:04:22.110773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:22.110773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:22.110773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 27 21:04:22.110773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 27 21:04:22.110773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 27 21:04:22.110773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 27 21:04:22.110773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 21:04:22.110773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application next(self.gen) Jun 27 21:04:22.111834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 27 21:04:22.111834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application yield resource Jun 27 21:04:22.111834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 21:04:22.111834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application next(self.gen) Jun 27 21:04:22.111834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 27 21:04:22.111834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self.session.rollback() Jun 27 21:04:22.111834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 27 21:04:22.111834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self.force_reraise() Jun 27 21:04:22.111834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 27 21:04:22.111834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise self.value Jun 27 21:04:22.111834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 27 21:04:22.111834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 27 21:04:22.111834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 27 21:04:22.111834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application session.commit() Jun 27 21:04:22.111834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 27 21:04:22.111834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 27 21:04:22.111834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 27 21:04:22.112729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._prepare_impl() Jun 27 21:04:22.112729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 27 21:04:22.112729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self.session.flush() Jun 27 21:04:22.112729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 27 21:04:22.112729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._flush(objects) Jun 27 21:04:22.112729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 27 21:04:22.112729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 27 21:04:22.112729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 27 21:04:22.112729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application compat.raise_( Jun 27 21:04:22.112729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 21:04:22.112729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception Jun 27 21:04:22.112729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 27 21:04:22.112729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application flush_context.execute() Jun 27 21:04:22.112729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 27 21:04:22.112729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rec.execute(self) Jun 27 21:04:22.112729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 27 21:04:22.112729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 27 21:04:22.113595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 27 21:04:22.113595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 27 21:04:22.113595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 27 21:04:22.113595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 27 21:04:22.113595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 27 21:04:22.113595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 27 21:04:22.113595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 27 21:04:22.113595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 27 21:04:22.113595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 27 21:04:22.113595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 27 21:04:22.113595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 27 21:04:22.113595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 27 21:04:22.113595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 27 21:04:22.113595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 27 21:04:22.113595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 21:04:22.113595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception Jun 27 21:04:22.114410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 21:04:22.114410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 21:04:22.114410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 21:04:22.114410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 21:04:22.114410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 21:04:22.114410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 21:04:22.114410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 21:04:22.114410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application conn.query(q) Jun 27 21:04:22.114410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 21:04:22.114410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 21:04:22.114410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 21:04:22.114410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application result.read() Jun 27 21:04:22.114410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 21:04:22.114410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 21:04:22.114410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 21:04:22.114410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 21:04:22.115272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 21:04:22.115272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 21:04:22.115272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 21:04:22.115272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 21:04:22.115272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 27 21:04:22.115272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 27 21:04:22.115272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ee44b7b99ce14843b76dbc528e4eb077', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 27 21:04:22.115272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 27 21:04:22.115272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:22.115272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:22.115272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:22.115272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:22.115272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:22.115272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:22.115272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:22.116176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:22.116176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:22.116176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:22.116176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:22.116176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:22.116176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:22.116176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:22.116176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 27 21:04:22.116176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 27 21:04:22.116176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:22.116176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:22.116176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:04:22.116176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:22.116176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:04:22.116176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:22.116176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 27 21:04:22.116176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 27 21:04:22.117085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 27 21:04:22.117085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 27 21:04:22.117085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 27 21:04:22.117085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 27 21:04:22.117085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 27 21:04:22.117085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:22.117085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 477/976] 10.222.0.27 () {60 vars in 1131 bytes} [Mon Jun 27 21:04:21 2022] POST /identity/v3/users => generated 160 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:22.117085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67709f1a-61f7-430e-b287-02bf1f73d65d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:22.125915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15cdbfe-73be-44ab-886a-823c05e818f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:22.126605 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cdbfe-73be-44ab-886a-823c05e818f9 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:22.126886 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cdbfe-73be-44ab-886a-823c05e818f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:22.127158 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cdbfe-73be-44ab-886a-823c05e818f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:22.128250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15cdbfe-73be-44ab-886a-823c05e818f9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:22.129744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15cdbfe-73be-44ab-886a-823c05e818f9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:22.133161 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:22.151096 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 500/977] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Jun 27 21:04:21 2022] POST /identity/v3/endpoints => generated 349 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:22.152094 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15cdbfe-73be-44ab-886a-823c05e818f9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:22.247730 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67709f1a-61f7-430e-b287-02bf1f73d65d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:22.276249 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:22.280553 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15cdbfe-73be-44ab-886a-823c05e818f9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:22.299163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9afdc-f57c-4b96-99a0-bc12eaea40b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:22.300555 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9afdc-f57c-4b96-99a0-bc12eaea40b2 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:22.300846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9afdc-f57c-4b96-99a0-bc12eaea40b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:22.301127 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9afdc-f57c-4b96-99a0-bc12eaea40b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:22.310815 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9afdc-f57c-4b96-99a0-bc12eaea40b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:22.312657 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9afdc-f57c-4b96-99a0-bc12eaea40b2 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:22.325619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd1d11-fc6a-438b-87f5-b548656f6dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:22.326803 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd1d11-fc6a-438b-87f5-b548656f6dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:22.327278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd1d11-fc6a-438b-87f5-b548656f6dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:22.327802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd1d11-fc6a-438b-87f5-b548656f6dd1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:22.329052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-c7e9afdc-f57c-4b96-99a0-bc12eaea40b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:22.329052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:22.329052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:22.329052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:22.329052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:22.329052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:22.329052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:22.329052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:22.329052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:22.329052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:22.329052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:22.329052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:22.329052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:04:22.329052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:04:22.329052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:04:22.329899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:04:22.329899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:22.329899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:22.329899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:22.329899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:22.329899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:22.329899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:22.329899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:22.329899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:22.329899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:22.329899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:22.329899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:22.329899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:22.329899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:22.329899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:22.329899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:22.329899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:22.330842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:04:22.330842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:04:22.330842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:04:22.330842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:04:22.330842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:04:22.330842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:04:22.330842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:22.330842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:22.330842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 478/978] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:04:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:22.338304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9afdc-f57c-4b96-99a0-bc12eaea40b2 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:22.345632 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd1d11-fc6a-438b-87f5-b548656f6dd1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:22.347120 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd1d11-fc6a-438b-87f5-b548656f6dd1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:22.364437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-98bd1d11-fc6a-438b-87f5-b548656f6dd1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 21:04:22.364437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:22.364437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:22.364437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:22.364437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:22.364437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:22.364437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:22.364437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:22.364437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:22.364437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:22.364437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:22.364437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:22.364437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:04:22.364437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:04:22.364437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:04:22.364437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:04:22.365373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:22.365373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:22.365373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:04:22.365373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:22.365373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:04:22.365373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:22.365373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:22.365373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:22.365373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:22.365373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:22.365373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:22.365373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:22.365373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:22.365373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:22.365373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:22.365373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:22.365373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:22.365373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:22.366291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:22.366291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:22.366291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:04:22.366291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:04:22.366291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:04:22.366291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:04:22.366291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:04:22.366291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:04:22.366291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 21:04:22.366291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:22.369811 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 501/979] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:04:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:22.372322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd1d11-fc6a-438b-87f5-b548656f6dd1 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:22.456083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:22.471317 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9afdc-f57c-4b96-99a0-bc12eaea40b2 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:22.492540 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd1d11-fc6a-438b-87f5-b548656f6dd1 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:22.516310 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c48750-e7e0-4297-b800-388fc694babd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:22.517753 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c48750-e7e0-4297-b800-388fc694babd None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:22.518258 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c48750-e7e0-4297-b800-388fc694babd None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:22.518987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c48750-e7e0-4297-b800-388fc694babd None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:22.520216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c48750-e7e0-4297-b800-388fc694babd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:22.521094 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c48750-e7e0-4297-b800-388fc694babd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:22.523201 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c48750-e7e0-4297-b800-388fc694babd None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:22.531163 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95ae15-1a32-4dec-a289-c61fb876bf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:22.531909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95ae15-1a32-4dec-a289-c61fb876bf11 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:22.532192 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95ae15-1a32-4dec-a289-c61fb876bf11 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:22.532810 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95ae15-1a32-4dec-a289-c61fb876bf11 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:22.533625 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95ae15-1a32-4dec-a289-c61fb876bf11 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:22.533924 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95ae15-1a32-4dec-a289-c61fb876bf11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:22.535373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95ae15-1a32-4dec-a289-c61fb876bf11 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:22.543244 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 479/980] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:04:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:22.549538 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c48750-e7e0-4297-b800-388fc694babd None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:22.587465 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 502/981] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Jun 27 21:04:22 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:22.595926 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:22.666552 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c48750-e7e0-4297-b800-388fc694babd None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:22.706022 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18aff8-d7af-4b8e-a3c3-265e2c0410e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:22.706716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18aff8-d7af-4b8e-a3c3-265e2c0410e7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:22.706997 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18aff8-d7af-4b8e-a3c3-265e2c0410e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:22.707270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18aff8-d7af-4b8e-a3c3-265e2c0410e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:22.708434 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18aff8-d7af-4b8e-a3c3-265e2c0410e7 None admin] RBAC: Adding query filter params (role.id=6134821b0f644a8ea7655b40f2ab3cc3, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=e802da0b078e457da466a9a7b0f15085) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:22.708764 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18aff8-d7af-4b8e-a3c3-265e2c0410e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:22.710228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18aff8-d7af-4b8e-a3c3-265e2c0410e7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:22.718081 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 480/982] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:04:22 2022] GET /identity/v3/role_assignments?user.id=e802da0b078e457da466a9a7b0f15085&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=6134821b0f644a8ea7655b40f2ab3cc3 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:23.003087 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce632e4-7063-48c2-b9e4-25b1a962c140 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:23.003347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce632e4-7063-48c2-b9e4-25b1a962c140 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:23.003657 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce632e4-7063-48c2-b9e4-25b1a962c140 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:23.005237 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 503/983] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:23.010446 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56effd8b-b4b1-44af-bdf6-dd3c469cc9fa None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:23.010716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56effd8b-b4b1-44af-bdf6-dd3c469cc9fa None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:23.010992 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56effd8b-b4b1-44af-bdf6-dd3c469cc9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:23.028982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-56effd8b-b4b1-44af-bdf6-dd3c469cc9fa None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:23.030488 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-56effd8b-b4b1-44af-bdf6-dd3c469cc9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:23.101175 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 481/984] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:23 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:23.123959 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce632e4-7063-48c2-b9e4-25b1a962c140 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:23.231010 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce632e4-7063-48c2-b9e4-25b1a962c140 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:23.233458 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:23.266715 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a067a8d-b424-42a4-99d1-a0d05b18496c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:23.267524 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a067a8d-b424-42a4-99d1-a0d05b18496c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:23.267829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a067a8d-b424-42a4-99d1-a0d05b18496c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:23.268096 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a067a8d-b424-42a4-99d1-a0d05b18496c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:23.268744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a067a8d-b424-42a4-99d1-a0d05b18496c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:23.270122 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a067a8d-b424-42a4-99d1-a0d05b18496c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:23.285424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-2a067a8d-b424-42a4-99d1-a0d05b18496c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 21:04:23.285424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:23.285424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:23.285424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:23.285424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:23.285424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:23.285424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:23.285424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:23.285424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 21:04:23.285424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:23.285424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:23.285424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:23.285424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:23.285424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:23.285424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:23.285424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:23.285424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:23.286480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:23.286480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:23.286480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:23.286480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:23.286480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:23.286480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:23.286480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:23.286480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:23.286480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:23.286480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:23.286480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:23.286480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:23.286480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:23.286480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:23.286480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:23.286480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:23.287311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:23.287311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:23.287311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:23.287311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:23.287311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:23.287311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:23.287311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:23.287311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:23.287311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:23.287311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:23.287311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:23.287311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:23.287311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 21:04:23.287311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:23.287311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 504/985] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:04:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:23.294967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56effd8b-b4b1-44af-bdf6-dd3c469cc9fa None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:23.333204 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:23.411334 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56effd8b-b4b1-44af-bdf6-dd3c469cc9fa None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:23.446431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6575a-7c30-49a7-8e8e-69aedb40b666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:23.447103 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6575a-7c30-49a7-8e8e-69aedb40b666 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:23.447382 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6575a-7c30-49a7-8e8e-69aedb40b666 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:23.447662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6575a-7c30-49a7-8e8e-69aedb40b666 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:23.448385 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa6575a-7c30-49a7-8e8e-69aedb40b666 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:23.448690 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa6575a-7c30-49a7-8e8e-69aedb40b666 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:23.450051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa6575a-7c30-49a7-8e8e-69aedb40b666 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:23.455158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 482/986] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Jun 27 21:04:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:23.460676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a067a8d-b424-42a4-99d1-a0d05b18496c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:23.561905 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a067a8d-b424-42a4-99d1-a0d05b18496c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:23.594503 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f2756-a03f-46b2-8c85-dca36ce7c590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:23.595161 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f2756-a03f-46b2-8c85-dca36ce7c590 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:23.595448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f2756-a03f-46b2-8c85-dca36ce7c590 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:23.595731 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f2756-a03f-46b2-8c85-dca36ce7c590 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:23.596438 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f2756-a03f-46b2-8c85-dca36ce7c590 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:23.596741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f2756-a03f-46b2-8c85-dca36ce7c590 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:23.598118 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f2756-a03f-46b2-8c85-dca36ce7c590 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:23.603181 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 505/987] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Jun 27 21:04:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:23.608897 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6575a-7c30-49a7-8e8e-69aedb40b666 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:23.709611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:23.715049 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6575a-7c30-49a7-8e8e-69aedb40b666 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:23.751440 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67810897-9a2a-4825-a1ad-b83238ba6c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:23.752151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67810897-9a2a-4825-a1ad-b83238ba6c36 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:23.752435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67810897-9a2a-4825-a1ad-b83238ba6c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:23.752712 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67810897-9a2a-4825-a1ad-b83238ba6c36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:23.753502 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67810897-9a2a-4825-a1ad-b83238ba6c36 None admin] RBAC: Adding query filter params (interface=public, service_id=99e9bbf00053476c94d1dd3ad3abd8ee, region_id=RegionOne) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:23.753819 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67810897-9a2a-4825-a1ad-b83238ba6c36 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:23.755202 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67810897-9a2a-4825-a1ad-b83238ba6c36 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:23.760810 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 483/988] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Jun 27 21:04:23 2022] GET /identity/v3/endpoints?service_id=99e9bbf00053476c94d1dd3ad3abd8ee&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:23.766370 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f2756-a03f-46b2-8c85-dca36ce7c590 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:23.800512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:23.875270 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f2756-a03f-46b2-8c85-dca36ce7c590 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:23.911636 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b04a5-8b11-4ea5-8877-35575a07cc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:23.913885 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b04a5-8b11-4ea5-8877-35575a07cc68 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:23.914169 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b04a5-8b11-4ea5-8877-35575a07cc68 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:23.914597 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b04a5-8b11-4ea5-8877-35575a07cc68 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:23.915473 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726b04a5-8b11-4ea5-8877-35575a07cc68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:23.916892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726b04a5-8b11-4ea5-8877-35575a07cc68 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:23.924016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 506/989] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Jun 27 21:04:23 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:04:24.241072 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0507c6-ea6e-485a-bbcb-4fc49162bd92 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:24.241258 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0507c6-ea6e-485a-bbcb-4fc49162bd92 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:24.241520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0507c6-ea6e-485a-bbcb-4fc49162bd92 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:24.243188 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 484/990] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:24.248498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f34e23-ac19-4b60-beec-c81e838146a7 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:24.248953 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f34e23-ac19-4b60-beec-c81e838146a7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:24.249029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f34e23-ac19-4b60-beec-c81e838146a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:24.266751 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-54f34e23-ac19-4b60-beec-c81e838146a7 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:24.267969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-54f34e23-ac19-4b60-beec-c81e838146a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:24.336975 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 507/991] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:24 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:24.359422 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0507c6-ea6e-485a-bbcb-4fc49162bd92 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:24.480663 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0507c6-ea6e-485a-bbcb-4fc49162bd92 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:24.515124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956428a-3f5d-44b2-b395-fd5aa0b7e8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:24.516093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956428a-3f5d-44b2-b395-fd5aa0b7e8f6 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:24.516381 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956428a-3f5d-44b2-b395-fd5aa0b7e8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:24.516657 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956428a-3f5d-44b2-b395-fd5aa0b7e8f6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:24.517316 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956428a-3f5d-44b2-b395-fd5aa0b7e8f6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:24.518712 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956428a-3f5d-44b2-b395-fd5aa0b7e8f6 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:24.531114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-c956428a-3f5d-44b2-b395-fd5aa0b7e8f6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:04:24.531114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:24.531114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:24.531114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:24.531114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:24.531114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:24.531114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:24.531114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:24.531114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:04:24.531114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:24.531114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:24.531114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:24.531114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:24.531114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:24.531114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:24.531114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:24.531114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:24.532754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:24.532754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:24.532754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:24.532754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:24.532754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:24.532754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:24.532754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:24.532754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:24.532754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:24.532754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:24.532754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:24.532754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:24.532754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:24.532754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:24.532754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:24.532754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:24.533603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:24.533603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:24.533603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:24.533603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:24.533603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:24.533603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:24.533603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:24.533603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:24.533603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:24.533603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:24.533603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:24.533603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:24.533603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:04:24.533603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:24.533603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 485/992] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Jun 27 21:04:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:24.540379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f34e23-ac19-4b60-beec-c81e838146a7 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:24.568131 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:24.644042 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f34e23-ac19-4b60-beec-c81e838146a7 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:24.681004 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8070b02-0443-4a8c-b4e1-f3f11a26327f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:24.681684 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8070b02-0443-4a8c-b4e1-f3f11a26327f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:24.681967 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8070b02-0443-4a8c-b4e1-f3f11a26327f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:24.682242 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8070b02-0443-4a8c-b4e1-f3f11a26327f None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:24.682952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8070b02-0443-4a8c-b4e1-f3f11a26327f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:24.683262 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8070b02-0443-4a8c-b4e1-f3f11a26327f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:24.684695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8070b02-0443-4a8c-b4e1-f3f11a26327f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:24.690069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 508/993] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Jun 27 21:04:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:24.695782 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956428a-3f5d-44b2-b395-fd5aa0b7e8f6 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:24.726564 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:24.748149 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0fb27-eacf-4ec1-abd9-fdac91d79d72 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:24.748467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0fb27-eacf-4ec1-abd9-fdac91d79d72 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:24.748709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0fb27-eacf-4ec1-abd9-fdac91d79d72 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:24.751291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 509/994] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:04:24.755090 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77e65d-e951-48d5-b700-c22da145af54 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:24.755379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77e65d-e951-48d5-b700-c22da145af54 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:24.755682 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77e65d-e951-48d5-b700-c22da145af54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:24.773202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0f77e65d-e951-48d5-b700-c22da145af54 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:24.774341 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0f77e65d-e951-48d5-b700-c22da145af54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:24.807626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956428a-3f5d-44b2-b395-fd5aa0b7e8f6 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:24.842379 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b146d7b-0070-4931-9c08-ee80a316ca6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:24.843058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b146d7b-0070-4931-9c08-ee80a316ca6c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:24.843337 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b146d7b-0070-4931-9c08-ee80a316ca6c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:24.843631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b146d7b-0070-4931-9c08-ee80a316ca6c None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:24.844368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b146d7b-0070-4931-9c08-ee80a316ca6c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:24.844670 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b146d7b-0070-4931-9c08-ee80a316ca6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:24.846046 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b146d7b-0070-4931-9c08-ee80a316ca6c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:24.851196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 486/995] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Jun 27 21:04:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:24.856925 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b146d7b-0070-4931-9c08-ee80a316ca6c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:24.859228 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 510/996] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:24 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:24.881298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77e65d-e951-48d5-b700-c22da145af54 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:24.904459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:24.954592 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:24.966251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b146d7b-0070-4931-9c08-ee80a316ca6c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:24.999903 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77e65d-e951-48d5-b700-c22da145af54 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:25.007761 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e278765-de22-4ada-9ae1-262207eef854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:25.008519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e278765-de22-4ada-9ae1-262207eef854 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:25.008862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e278765-de22-4ada-9ae1-262207eef854 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:25.009143 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e278765-de22-4ada-9ae1-262207eef854 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:25.010101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e278765-de22-4ada-9ae1-262207eef854 None admin] RBAC: Adding query filter params (interface=admin, service_id=7f78daafca654921adf1ae5a10ca18c4, region_id=RegionOne) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:25.010420 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e278765-de22-4ada-9ae1-262207eef854 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:25.012033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e278765-de22-4ada-9ae1-262207eef854 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:25.025567 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 487/997] 10.222.0.27 () {56 vars in 1249 bytes} [Mon Jun 27 21:04:24 2022] GET /identity/v3/endpoints?service_id=7f78daafca654921adf1ae5a10ca18c4&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:25.026843 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e278765-de22-4ada-9ae1-262207eef854 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:25.047228 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27872eb-0721-481d-a7f6-ddb6cea1e226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:25.048181 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27872eb-0721-481d-a7f6-ddb6cea1e226 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:25.048524 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27872eb-0721-481d-a7f6-ddb6cea1e226 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:25.048806 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27872eb-0721-481d-a7f6-ddb6cea1e226 None admin] PATH_INFO: `/v3/services/image` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:25.049502 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27872eb-0721-481d-a7f6-ddb6cea1e226 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:25.050912 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27872eb-0721-481d-a7f6-ddb6cea1e226 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:25.063505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-d27872eb-0721-481d-a7f6-ddb6cea1e226 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 21:04:25.063505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:25.063505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:25.063505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:25.063505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:25.063505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:25.063505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:25.063505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:25.063505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 21:04:25.063505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:25.063505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:25.063505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:25.063505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:25.063505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:25.063505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:25.063505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:25.063505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:25.064638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:25.064638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:25.064638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:25.064638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:25.064638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:25.064638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:25.064638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:25.064638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:25.064638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:25.064638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:25.064638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:25.064638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:25.064638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:25.064638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:25.064638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:25.064638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:25.065449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:25.065449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:25.065449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:25.065449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:25.065449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:25.065449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:25.065449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:25.065449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:25.065449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:25.065449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:25.065449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:25.065449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:25.065449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 21:04:25.065449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:25.069863 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 511/998] 10.222.0.27 () {56 vars in 1105 bytes} [Mon Jun 27 21:04:24 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:25.070673 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27872eb-0721-481d-a7f6-ddb6cea1e226 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:25.146607 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e278765-de22-4ada-9ae1-262207eef854 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:25.159090 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:25.175943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27872eb-0721-481d-a7f6-ddb6cea1e226 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:25.184966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50a953-19e7-4be8-bd39-12b574991f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:25.185985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50a953-19e7-4be8-bd39-12b574991f26 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:25.186270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50a953-19e7-4be8-bd39-12b574991f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:25.186553 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50a953-19e7-4be8-bd39-12b574991f26 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:25.187545 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50a953-19e7-4be8-bd39-12b574991f26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:25.189247 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50a953-19e7-4be8-bd39-12b574991f26 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:25.195693 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 488/999] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Jun 27 21:04:25 2022] GET /identity/v3/services => generated 1727 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:04:25.210678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026b310-c200-4d1e-a33f-5e48e84d98ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:25.211538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026b310-c200-4d1e-a33f-5e48e84d98ff None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:25.211950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026b310-c200-4d1e-a33f-5e48e84d98ff None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:25.212231 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026b310-c200-4d1e-a33f-5e48e84d98ff None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:25.212956 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026b310-c200-4d1e-a33f-5e48e84d98ff None admin] RBAC: Adding query filter params (name=image) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:25.213262 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026b310-c200-4d1e-a33f-5e48e84d98ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:25.214816 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026b310-c200-4d1e-a33f-5e48e84d98ff None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:25.225540 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 512/1000] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:25.226663 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50a953-19e7-4be8-bd39-12b574991f26 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:25.249410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:25.340022 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50a953-19e7-4be8-bd39-12b574991f26 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:25.377004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a82b5d-1a26-4dda-ba6a-305cf83b6ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:25.377765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a82b5d-1a26-4dda-ba6a-305cf83b6ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:25.378143 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a82b5d-1a26-4dda-ba6a-305cf83b6ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:25.378424 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a82b5d-1a26-4dda-ba6a-305cf83b6ff4 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:25.379144 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a82b5d-1a26-4dda-ba6a-305cf83b6ff4 None admin] RBAC: Adding query filter params (type=image) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:25.379453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a82b5d-1a26-4dda-ba6a-305cf83b6ff4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:25.380899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a82b5d-1a26-4dda-ba6a-305cf83b6ff4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:25.386273 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 489/1001] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:25 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:25.952696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9e536-12ed-4f87-bdd2-7760705dc4c6 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:25.953275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9e536-12ed-4f87-bdd2-7760705dc4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:25.953275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9e536-12ed-4f87-bdd2-7760705dc4c6 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:25.954822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 513/1002] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:25.960027 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22727113-10be-443a-9eb4-cd92e43f1819 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:25.960293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22727113-10be-443a-9eb4-cd92e43f1819 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:25.960599 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22727113-10be-443a-9eb4-cd92e43f1819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:25.980042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-22727113-10be-443a-9eb4-cd92e43f1819 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:25.981268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-22727113-10be-443a-9eb4-cd92e43f1819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:26.053208 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 490/1003] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:26.076199 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9e536-12ed-4f87-bdd2-7760705dc4c6 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:26.187340 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9e536-12ed-4f87-bdd2-7760705dc4c6 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:26.220558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a5d4e-2972-4760-bb47-4b161c6e0ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:26.221288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a5d4e-2972-4760-bb47-4b161c6e0ecc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:26.221564 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a5d4e-2972-4760-bb47-4b161c6e0ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:26.221869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a5d4e-2972-4760-bb47-4b161c6e0ecc None admin] PATH_INFO: `/v3/services/compute` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:26.222489 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a5d4e-2972-4760-bb47-4b161c6e0ecc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:26.223897 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a5d4e-2972-4760-bb47-4b161c6e0ecc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:26.235475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-869a5d4e-2972-4760-bb47-4b161c6e0ecc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 21:04:26.235475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:26.235475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:26.235475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:26.235475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:26.235475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:26.235475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:26.235475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:26.235475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 21:04:26.235475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:26.235475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:26.235475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:26.235475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:26.235475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:26.235475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:26.235475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:26.235475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:26.236642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:26.236642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:26.236642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:26.236642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:26.236642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:26.236642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:26.236642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:26.236642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:26.236642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:26.236642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:26.236642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:26.236642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:26.236642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:26.236642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:26.236642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:26.236642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:26.237510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:26.237510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:26.237510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:26.237510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:26.237510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:26.237510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:26.237510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:26.237510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:26.237510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:26.237510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:26.237510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:26.237510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:26.237510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 21:04:26.237510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:26.237510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 514/1004] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:04:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:26.244820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22727113-10be-443a-9eb4-cd92e43f1819 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:26.261325 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:26.352987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22727113-10be-443a-9eb4-cd92e43f1819 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:26.387582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4081e6-8b6d-43d6-8f97-b0bfe356671c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:26.388410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4081e6-8b6d-43d6-8f97-b0bfe356671c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:26.388581 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4081e6-8b6d-43d6-8f97-b0bfe356671c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:26.388836 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4081e6-8b6d-43d6-8f97-b0bfe356671c None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:26.389565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4081e6-8b6d-43d6-8f97-b0bfe356671c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:26.389871 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4081e6-8b6d-43d6-8f97-b0bfe356671c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:26.391308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4081e6-8b6d-43d6-8f97-b0bfe356671c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:26.396647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 491/1005] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Jun 27 21:04:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:26.402178 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a5d4e-2972-4760-bb47-4b161c6e0ecc None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:26.418990 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:26.510579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a5d4e-2972-4760-bb47-4b161c6e0ecc None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:26.544092 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7f199-576a-4765-9aeb-da79b945a9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:26.544758 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7f199-576a-4765-9aeb-da79b945a9ae None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:26.545038 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7f199-576a-4765-9aeb-da79b945a9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:26.545305 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7f199-576a-4765-9aeb-da79b945a9ae None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:26.546006 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7f199-576a-4765-9aeb-da79b945a9ae None admin] RBAC: Adding query filter params (type=compute) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:26.546307 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7f199-576a-4765-9aeb-da79b945a9ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:26.547735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7f199-576a-4765-9aeb-da79b945a9ae None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:26.552817 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 515/1006] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Jun 27 21:04:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:26.558574 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4081e6-8b6d-43d6-8f97-b0bfe356671c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:26.634805 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:26.659301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4081e6-8b6d-43d6-8f97-b0bfe356671c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:26.692867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260614df-3de4-4437-9f3f-0fe89f107d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:26.693526 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260614df-3de4-4437-9f3f-0fe89f107d8d None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:26.693806 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260614df-3de4-4437-9f3f-0fe89f107d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:26.694085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260614df-3de4-4437-9f3f-0fe89f107d8d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:26.694831 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260614df-3de4-4437-9f3f-0fe89f107d8d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:26.696234 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260614df-3de4-4437-9f3f-0fe89f107d8d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:26.717659 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 492/1007] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Jun 27 21:04:26 2022] POST /identity/v3/endpoints => generated 334 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:27.220666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33e92b-ba9f-4e6d-bd68-d72bc4dd7664 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:27.221244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33e92b-ba9f-4e6d-bd68-d72bc4dd7664 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:27.221244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33e92b-ba9f-4e6d-bd68-d72bc4dd7664 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:27.222568 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 516/1008] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:27.228337 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d2e9a-b06d-40b0-81f0-6796a751dfa8 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:27.228591 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d2e9a-b06d-40b0-81f0-6796a751dfa8 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:27.228902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d2e9a-b06d-40b0-81f0-6796a751dfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:27.247006 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-573d2e9a-b06d-40b0-81f0-6796a751dfa8 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:27.248159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-573d2e9a-b06d-40b0-81f0-6796a751dfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:27.326309 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 493/1009] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:27 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:27.348938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33e92b-ba9f-4e6d-bd68-d72bc4dd7664 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:27.400139 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d641f-2c2a-4db5-94b1-d51a8b68fd7f None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:27.400678 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d641f-2c2a-4db5-94b1-d51a8b68fd7f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:27.400678 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d641f-2c2a-4db5-94b1-d51a8b68fd7f None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:27.402225 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 494/1010] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:04:27.407241 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fa8da-f244-4b8b-ac72-9f593b1309c1 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:27.407933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fa8da-f244-4b8b-ac72-9f593b1309c1 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:27.408184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fa8da-f244-4b8b-ac72-9f593b1309c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:27.425011 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-cb7fa8da-f244-4b8b-ac72-9f593b1309c1 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:27.426217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-cb7fa8da-f244-4b8b-ac72-9f593b1309c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:27.431215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:27.457645 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33e92b-ba9f-4e6d-bd68-d72bc4dd7664 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:27.481162 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:27.494626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba4f86-22aa-4676-bf05-f3aa836c3206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:27.495458 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba4f86-22aa-4676-bf05-f3aa836c3206 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:27.495782 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba4f86-22aa-4676-bf05-f3aa836c3206 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:27.496063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba4f86-22aa-4676-bf05-f3aa836c3206 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:27.496722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba4f86-22aa-4676-bf05-f3aa836c3206 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:27.498172 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba4f86-22aa-4676-bf05-f3aa836c3206 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:27.498528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 495/1011] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:27 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:27.512102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-45ba4f86-22aa-4676-bf05-f3aa836c3206 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:04:27.512102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:27.512102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:27.512102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:27.512102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:27.512102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:27.512102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:27.512102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:27.512102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:04:27.512102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:27.512102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:27.512102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:27.512102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:27.512102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:27.512102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:27.512102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:27.512102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:27.513102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:27.513102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:27.513102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:27.513102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:27.513102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:27.513102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:27.513102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:27.513102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:27.513102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:27.513102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:27.513102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:27.513102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:27.513102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:27.513102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:27.513102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:27.513102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:27.513930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:27.513930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:27.513930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:27.513930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:27.513930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:27.513930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:27.513930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:27.513930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:27.513930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:27.513930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:27.513930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:27.513930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:27.513930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:04:27.513930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:27.515438 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 517/1012] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Jun 27 21:04:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:27.519268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fa8da-f244-4b8b-ac72-9f593b1309c1 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:27.521372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba4f86-22aa-4676-bf05-f3aa836c3206 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:27.626228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fa8da-f244-4b8b-ac72-9f593b1309c1 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:27.629139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba4f86-22aa-4676-bf05-f3aa836c3206 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:27.661446 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cbe6e-9a79-4b47-b1f7-e675820cefcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:27.662136 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cbe6e-9a79-4b47-b1f7-e675820cefcb None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:27.662394 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cbe6e-9a79-4b47-b1f7-e675820cefcb None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:27.662671 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cbe6e-9a79-4b47-b1f7-e675820cefcb None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:27.663383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8cbe6e-9a79-4b47-b1f7-e675820cefcb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:27.663727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8cbe6e-9a79-4b47-b1f7-e675820cefcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:27.665109 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8cbe6e-9a79-4b47-b1f7-e675820cefcb None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:27.666873 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174fce2-85f4-4651-b451-c2739cd931c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:27.667798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174fce2-85f4-4651-b451-c2739cd931c9 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:27.668286 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174fce2-85f4-4651-b451-c2739cd931c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:27.668761 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174fce2-85f4-4651-b451-c2739cd931c9 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:27.669660 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174fce2-85f4-4651-b451-c2739cd931c9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:27.669841 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 496/1013] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Jun 27 21:04:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:27.671523 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174fce2-85f4-4651-b451-c2739cd931c9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:27.675480 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cbe6e-9a79-4b47-b1f7-e675820cefcb None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:27.681891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:27.688300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 518/1014] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Jun 27 21:04:27 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:27.734602 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:27.787473 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cbe6e-9a79-4b47-b1f7-e675820cefcb None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:27.831240 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207c957-9cfc-4ce8-b460-dbeff98f9407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:27.831962 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207c957-9cfc-4ce8-b460-dbeff98f9407 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:27.832248 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207c957-9cfc-4ce8-b460-dbeff98f9407 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:27.832525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207c957-9cfc-4ce8-b460-dbeff98f9407 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:27.833241 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207c957-9cfc-4ce8-b460-dbeff98f9407 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:27.833551 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207c957-9cfc-4ce8-b460-dbeff98f9407 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:27.834934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207c957-9cfc-4ce8-b460-dbeff98f9407 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:27.840641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 497/1015] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Jun 27 21:04:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:27.846210 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174fce2-85f4-4651-b451-c2739cd931c9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:27.949640 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174fce2-85f4-4651-b451-c2739cd931c9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:27.985044 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c86266-e84e-4727-a8dd-86bbb9b775b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:27.985716 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c86266-e84e-4727-a8dd-86bbb9b775b3 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:27.985976 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c86266-e84e-4727-a8dd-86bbb9b775b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:27.986272 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c86266-e84e-4727-a8dd-86bbb9b775b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:27.987029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c86266-e84e-4727-a8dd-86bbb9b775b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:27.991036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c86266-e84e-4727-a8dd-86bbb9b775b3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:28.008065 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 519/1016] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Jun 27 21:04:27 2022] POST /identity/v3/endpoints => generated 325 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:28.876748 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fdadd-6c17-4ef3-bb38-94f5f5183278 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:28.877284 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fdadd-6c17-4ef3-bb38-94f5f5183278 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:28.877284 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fdadd-6c17-4ef3-bb38-94f5f5183278 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:28.878690 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 498/1017] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:28.884062 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf94703-a823-47cf-8c3b-d2b0e3dedd9f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:28.884479 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf94703-a823-47cf-8c3b-d2b0e3dedd9f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:28.884647 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf94703-a823-47cf-8c3b-d2b0e3dedd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:28.901775 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-acf94703-a823-47cf-8c3b-d2b0e3dedd9f None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:28.902969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-acf94703-a823-47cf-8c3b-d2b0e3dedd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:28.962014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:28.984691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 520/1018] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:29.008477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fdadd-6c17-4ef3-bb38-94f5f5183278 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:29.117342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fdadd-6c17-4ef3-bb38-94f5f5183278 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:29.131807 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:29.155485 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b430540-9b63-46c7-a6de-aed42d773309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:29.156449 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b430540-9b63-46c7-a6de-aed42d773309 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:29.156737 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b430540-9b63-46c7-a6de-aed42d773309 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:29.157202 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b430540-9b63-46c7-a6de-aed42d773309 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:29.168257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b430540-9b63-46c7-a6de-aed42d773309 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:29.176631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-9b430540-9b63-46c7-a6de-aed42d773309 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:04:29.176631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:29.176631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:29.176631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:29.176631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:29.176631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:29.176631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:29.176631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:29.176631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:29.176631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:29.176631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:29.176631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:29.176631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:29.176631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:29.176631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:29.176631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:29.177568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:29.177568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:29.177568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:29.177568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:29.177568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:29.177568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:29.177568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:29.177568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:29.177568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:29.177568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:29.177568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:29.177568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:29.177568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:29.177568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:29.177568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:29.177568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:29.178408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:29.178408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:29.178408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:29.178408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:29.178408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:29.178408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:29.178408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:29.178408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:29.178408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:29.178408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:29.178408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:04:29.178408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:29.178408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 499/1019] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Jun 27 21:04:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:29.183900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf94703-a823-47cf-8c3b-d2b0e3dedd9f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:29.288243 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf94703-a823-47cf-8c3b-d2b0e3dedd9f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:29.324256 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9f923-1440-4122-ac65-1a1cce8c9c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:29.324985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9f923-1440-4122-ac65-1a1cce8c9c05 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:29.325271 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9f923-1440-4122-ac65-1a1cce8c9c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:29.325548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9f923-1440-4122-ac65-1a1cce8c9c05 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:29.326302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9f923-1440-4122-ac65-1a1cce8c9c05 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:29.326615 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9f923-1440-4122-ac65-1a1cce8c9c05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:29.328026 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9f923-1440-4122-ac65-1a1cce8c9c05 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:29.336531 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 521/1020] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Jun 27 21:04:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:29.342109 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b430540-9b63-46c7-a6de-aed42d773309 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:29.445080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b430540-9b63-46c7-a6de-aed42d773309 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:29.478945 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125a666-c8b7-4586-8ab3-60732f15a90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:29.479789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125a666-c8b7-4586-8ab3-60732f15a90b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:29.480075 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125a666-c8b7-4586-8ab3-60732f15a90b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:29.480348 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125a666-c8b7-4586-8ab3-60732f15a90b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:29.491253 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9125a666-c8b7-4586-8ab3-60732f15a90b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:29.492768 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9125a666-c8b7-4586-8ab3-60732f15a90b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:29.510341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-9125a666-c8b7-4586-8ab3-60732f15a90b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:29.510341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:29.510341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:29.510341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:29.510341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:29.510341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:29.510341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:29.510341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:29.510341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:29.510341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:29.510341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:29.510341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:29.510341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:29.510341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:29.510341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:29.511260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:29.511260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:29.511260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:29.511260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:29.511260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:29.511260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:29.511260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:29.511260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:29.511260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:29.511260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:29.511260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:29.511260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:29.511260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:29.511260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:29.511260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:29.511260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:29.511260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:29.512212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:29.512212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:29.512212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:29.512212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:29.515692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 500/1021] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:29.517417 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9f923-1440-4122-ac65-1a1cce8c9c05 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:29.569618 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:29.622501 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9f923-1440-4122-ac65-1a1cce8c9c05 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:29.656612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05c631-9d64-4867-8ae0-d253830b7ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:29.657293 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05c631-9d64-4867-8ae0-d253830b7ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:29.657938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05c631-9d64-4867-8ae0-d253830b7ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:29.658214 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05c631-9d64-4867-8ae0-d253830b7ad1 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:29.658939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05c631-9d64-4867-8ae0-d253830b7ad1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:29.659250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05c631-9d64-4867-8ae0-d253830b7ad1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:29.660672 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05c631-9d64-4867-8ae0-d253830b7ad1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:29.672839 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 522/1022] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:29.678305 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125a666-c8b7-4586-8ab3-60732f15a90b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:29.705779 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:29.736518 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897341d-3cfe-44a9-948f-ff5a10fdbaa1 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:29.736879 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897341d-3cfe-44a9-948f-ff5a10fdbaa1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:29.737084 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897341d-3cfe-44a9-948f-ff5a10fdbaa1 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:29.738477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 523/1023] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:04:29.743365 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa6b10-556f-4ca1-a997-4a70cfe608e9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:29.743678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa6b10-556f-4ca1-a997-4a70cfe608e9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:29.743973 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa6b10-556f-4ca1-a997-4a70cfe608e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:29.760758 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6ffa6b10-556f-4ca1-a997-4a70cfe608e9 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:29.761962 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6ffa6b10-556f-4ca1-a997-4a70cfe608e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:29.785658 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125a666-c8b7-4586-8ab3-60732f15a90b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:29.820950 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee99704-e7bb-4414-a824-d76f09ebe64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:29.821753 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee99704-e7bb-4414-a824-d76f09ebe64c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:29.822036 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee99704-e7bb-4414-a824-d76f09ebe64c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:29.822348 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee99704-e7bb-4414-a824-d76f09ebe64c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:29.830204 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee99704-e7bb-4414-a824-d76f09ebe64c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:29.831683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee99704-e7bb-4414-a824-d76f09ebe64c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:29.836324 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 524/1024] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:29.845825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-cee99704-e7bb-4414-a824-d76f09ebe64c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:04:29.845825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:29.845825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:29.845825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:29.845825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:29.845825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:29.845825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:29.845825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:29.845825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:29.845825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:29.845825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:29.845825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:29.845825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:04:29.845825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:04:29.845825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:04:29.845825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:04:29.846750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:29.846750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:29.846750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:04:29.846750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:29.846750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:04:29.846750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:29.846750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:29.846750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:29.846750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:29.846750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:29.846750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:29.846750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:29.846750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:29.846750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:29.846750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:29.846750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:29.846750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:29.846750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:29.847763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:29.847763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:29.847763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:04:29.847763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:04:29.847763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:04:29.847763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:04:29.847763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:04:29.847763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:04:29.847763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:04:29.847763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:29.847763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 501/1025] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Jun 27 21:04:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:29.852813 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa6b10-556f-4ca1-a997-4a70cfe608e9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:29.860834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee99704-e7bb-4414-a824-d76f09ebe64c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:29.894419 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:29.948554 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:29.959428 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa6b10-556f-4ca1-a997-4a70cfe608e9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:29.966992 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee99704-e7bb-4414-a824-d76f09ebe64c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:30.004357 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fe068-0350-459f-8963-7687e184fbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:30.005054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fe068-0350-459f-8963-7687e184fbb5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:30.005363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fe068-0350-459f-8963-7687e184fbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:30.005615 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fe068-0350-459f-8963-7687e184fbb5 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:30.006460 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6fe068-0350-459f-8963-7687e184fbb5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:30.006770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6fe068-0350-459f-8963-7687e184fbb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:30.008260 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6fe068-0350-459f-8963-7687e184fbb5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:30.008733 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67081e20-79ea-4daa-9085-37efb7a01aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:30.009542 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67081e20-79ea-4daa-9085-37efb7a01aed None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:30.009935 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67081e20-79ea-4daa-9085-37efb7a01aed None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:30.010210 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67081e20-79ea-4daa-9085-37efb7a01aed None admin] PATH_INFO: `/v3/services/image` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:30.010974 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67081e20-79ea-4daa-9085-37efb7a01aed None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:30.012399 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67081e20-79ea-4daa-9085-37efb7a01aed None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:30.024794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-67081e20-79ea-4daa-9085-37efb7a01aed None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 21:04:30.024794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:30.024794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:30.024794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:30.024794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:30.024794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:30.024794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:30.024794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:30.024794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 21:04:30.024794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:30.024794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:30.024794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:30.024794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:30.024794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:30.024794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:30.024794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:30.024794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:30.030735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:30.030735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:30.030735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:30.030735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:30.030735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:30.030735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:30.030735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:30.030735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:30.030735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:30.030735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:30.030735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:30.030735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:30.030735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:30.030735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:30.030735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:30.030735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:30.031850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:30.031850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:30.031850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:30.031850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:30.031850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:30.031850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:30.031850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:30.031850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:30.031850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:30.031850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:30.031850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:30.031850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:30.031850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 21:04:30.031850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:30.031850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 502/1026] 10.222.0.27 () {56 vars in 1105 bytes} [Mon Jun 27 21:04:29 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:30.033187 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d3af6-7b57-4c4f-9bd5-7fb0e694ccc5 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:30.033268 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 525/1027] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Jun 27 21:04:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:30.033361 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d3af6-7b57-4c4f-9bd5-7fb0e694ccc5 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:30.033361 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d3af6-7b57-4c4f-9bd5-7fb0e694ccc5 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:30.033361 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 503/1028] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:30.033665 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fe068-0350-459f-8963-7687e184fbb5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:30.034538 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d3af6-7b57-4c4f-9bd5-7fb0e694ccc5 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:30.125066 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:30.131316 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fe068-0350-459f-8963-7687e184fbb5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:30.133968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d3af6-7b57-4c4f-9bd5-7fb0e694ccc5 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:30.167077 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b8d60-8adb-4a65-b28c-1524e5842860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:30.167736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b270f9-0d22-45f7-b92f-5197c9d2194f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:30.167849 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b8d60-8adb-4a65-b28c-1524e5842860 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:30.168043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b8d60-8adb-4a65-b28c-1524e5842860 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:30.168319 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b8d60-8adb-4a65-b28c-1524e5842860 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:30.168417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b270f9-0d22-45f7-b92f-5197c9d2194f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:30.168681 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b270f9-0d22-45f7-b92f-5197c9d2194f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:30.168963 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b270f9-0d22-45f7-b92f-5197c9d2194f None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:30.169668 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b270f9-0d22-45f7-b92f-5197c9d2194f None admin] RBAC: Adding query filter params (name=image) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:30.169975 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b270f9-0d22-45f7-b92f-5197c9d2194f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:30.171372 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b270f9-0d22-45f7-b92f-5197c9d2194f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:30.176630 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 504/1029] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:30.177881 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e036c5-8f0b-41cd-aa38-68e49f9a74a5 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:30.178169 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e036c5-8f0b-41cd-aa38-68e49f9a74a5 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:30.178451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e036c5-8f0b-41cd-aa38-68e49f9a74a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:30.194774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-35e036c5-8f0b-41cd-aa38-68e49f9a74a5 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:30.196017 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-35e036c5-8f0b-41cd-aa38-68e49f9a74a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:30.203298 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:30.239917 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b8d60-8adb-4a65-b28c-1524e5842860 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:30.241316 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b8d60-8adb-4a65-b28c-1524e5842860 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:30.266007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 505/1030] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:30.266837 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e036c5-8f0b-41cd-aa38-68e49f9a74a5 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:30.336724 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:30.343934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 526/1031] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:30 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:30.344738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b8d60-8adb-4a65-b28c-1524e5842860 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:30.372244 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e036c5-8f0b-41cd-aa38-68e49f9a74a5 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:30.407862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1203b-7beb-4225-a3d6-fb260115d577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:30.408538 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1203b-7beb-4225-a3d6-fb260115d577 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:30.409190 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1203b-7beb-4225-a3d6-fb260115d577 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:30.409190 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1203b-7beb-4225-a3d6-fb260115d577 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:30.409792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1203b-7beb-4225-a3d6-fb260115d577 None admin] RBAC: Adding query filter params (type=image) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:30.410174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1203b-7beb-4225-a3d6-fb260115d577 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:30.411493 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1203b-7beb-4225-a3d6-fb260115d577 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:30.416557 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 506/1032] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:30.417145 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1203b-7beb-4225-a3d6-fb260115d577 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:30.425384 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:30.443910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b8d60-8adb-4a65-b28c-1524e5842860 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:30.481624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f44d69-7903-4d10-b695-36f418d8ba61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:30.482452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f44d69-7903-4d10-b695-36f418d8ba61 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:30.482724 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f44d69-7903-4d10-b695-36f418d8ba61 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:30.482990 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f44d69-7903-4d10-b695-36f418d8ba61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:30.484199 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f44d69-7903-4d10-b695-36f418d8ba61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:30.485668 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f44d69-7903-4d10-b695-36f418d8ba61 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:30.487277 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 527/1033] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:30.488134 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f44d69-7903-4d10-b695-36f418d8ba61 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:30.522868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1203b-7beb-4225-a3d6-fb260115d577 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:30.550132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:30.557509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563823f-f218-4939-a87f-ee9f8c44695c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:30.558335 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563823f-f218-4939-a87f-ee9f8c44695c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:30.558611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563823f-f218-4939-a87f-ee9f8c44695c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:30.558889 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563823f-f218-4939-a87f-ee9f8c44695c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:30.569201 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563823f-f218-4939-a87f-ee9f8c44695c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:30.570661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563823f-f218-4939-a87f-ee9f8c44695c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:30.587747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f44d69-7903-4d10-b695-36f418d8ba61 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:30.588196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-c563823f-f218-4939-a87f-ee9f8c44695c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:30.588196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:30.588196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:30.588196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:30.588196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:30.588196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:30.588196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:30.588196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:30.588196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:30.588196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:30.588196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:30.588196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:30.588196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:30.588196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:30.588196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:30.589073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:30.589073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:30.589073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:30.589073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:30.589073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:30.589073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:30.589073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:30.589073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:30.589073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:30.589073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:30.589073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:30.589073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:30.589073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:30.589073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:30.589073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:30.589073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:30.589073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:30.589955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:30.589955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:30.589955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:30.589955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:30.589955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 507/1034] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:30.590655 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563823f-f218-4939-a87f-ee9f8c44695c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:30.623270 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03a32e-b026-454e-8f0f-ecb8b8906ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:30.623978 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03a32e-b026-454e-8f0f-ecb8b8906ead None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:30.624259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03a32e-b026-454e-8f0f-ecb8b8906ead None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:30.624537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03a32e-b026-454e-8f0f-ecb8b8906ead None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:30.625330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03a32e-b026-454e-8f0f-ecb8b8906ead None admin] RBAC: Adding query filter params (interface=public, service_id=093e724f1a85483ea4ee23ced11fb0fb, region_id=RegionOne) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:30.625641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03a32e-b026-454e-8f0f-ecb8b8906ead None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:30.627017 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03a32e-b026-454e-8f0f-ecb8b8906ead None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:30.634312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 528/1035] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Jun 27 21:04:30 2022] GET /identity/v3/endpoints?service_id=093e724f1a85483ea4ee23ced11fb0fb&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:30.635079 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03a32e-b026-454e-8f0f-ecb8b8906ead None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:30.648948 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:30.691474 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563823f-f218-4939-a87f-ee9f8c44695c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:30.726265 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12563486-a0ec-4bfa-a278-9040650ac274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:30.726872 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12563486-a0ec-4bfa-a278-9040650ac274 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:30.727153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12563486-a0ec-4bfa-a278-9040650ac274 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:30.727426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12563486-a0ec-4bfa-a278-9040650ac274 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:30.728170 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12563486-a0ec-4bfa-a278-9040650ac274 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:30.729610 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12563486-a0ec-4bfa-a278-9040650ac274 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:30.734400 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03a32e-b026-454e-8f0f-ecb8b8906ead None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:30.758696 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 508/1036] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Jun 27 21:04:30 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:30.760002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12563486-a0ec-4bfa-a278-9040650ac274 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:30.767566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:30.776964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537888f4-b9bf-4df1-8d11-f392e558f70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:30.777570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537888f4-b9bf-4df1-8d11-f392e558f70d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:30.777832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537888f4-b9bf-4df1-8d11-f392e558f70d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:30.778117 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537888f4-b9bf-4df1-8d11-f392e558f70d None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:30.778868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537888f4-b9bf-4df1-8d11-f392e558f70d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:30.779159 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537888f4-b9bf-4df1-8d11-f392e558f70d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:30.780655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537888f4-b9bf-4df1-8d11-f392e558f70d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:30.795118 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 529/1037] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:30.801215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537888f4-b9bf-4df1-8d11-f392e558f70d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:30.898291 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12563486-a0ec-4bfa-a278-9040650ac274 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:30.911384 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:30.915782 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537888f4-b9bf-4df1-8d11-f392e558f70d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:30.944381 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690838dd-6fb2-495d-b2e3-49224e645013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:30.945132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690838dd-6fb2-495d-b2e3-49224e645013 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:30.945441 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690838dd-6fb2-495d-b2e3-49224e645013 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:30.945710 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690838dd-6fb2-495d-b2e3-49224e645013 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:30.946403 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690838dd-6fb2-495d-b2e3-49224e645013 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:30.947812 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690838dd-6fb2-495d-b2e3-49224e645013 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:30.954805 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 509/1038] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Jun 27 21:04:30 2022] GET /identity/v3/services => generated 1957 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:04:30.956978 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425f601-c417-4441-b4b5-58d491588248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:30.957798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425f601-c417-4441-b4b5-58d491588248 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:30.958063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425f601-c417-4441-b4b5-58d491588248 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:30.958338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425f601-c417-4441-b4b5-58d491588248 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:30.966041 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7425f601-c417-4441-b4b5-58d491588248 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:30.967562 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7425f601-c417-4441-b4b5-58d491588248 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:30.984031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-7425f601-c417-4441-b4b5-58d491588248 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:30.984031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:30.984031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:30.984031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:30.984031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:30.984031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:30.984031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:30.984031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:30.984031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:30.984031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:30.984031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:30.984031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:30.984031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:04:30.984031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:04:30.984031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:04:30.985126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:04:30.985126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:30.985126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:30.985126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:30.985126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:30.985126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:30.985126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:30.985126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:30.985126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:30.985126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:30.985126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:30.985126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:30.985126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:30.985126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:30.985126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:30.985126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:30.985126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:30.986014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:04:30.986014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:04:30.986014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:04:30.986014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:04:30.986014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:04:30.986014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:04:30.986014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:30.986014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:30.986014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 530/1039] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:04:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:30.994179 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690838dd-6fb2-495d-b2e3-49224e645013 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:31.115342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690838dd-6fb2-495d-b2e3-49224e645013 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:31.154031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d2214-1e71-4f82-beaa-712d4833eb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:31.154714 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d2214-1e71-4f82-beaa-712d4833eb7f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:31.154999 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d2214-1e71-4f82-beaa-712d4833eb7f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:31.155273 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d2214-1e71-4f82-beaa-712d4833eb7f None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:31.156174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d2214-1e71-4f82-beaa-712d4833eb7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:31.156532 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d2214-1e71-4f82-beaa-712d4833eb7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:31.158317 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d2214-1e71-4f82-beaa-712d4833eb7f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:31.172604 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 510/1040] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:04:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:31.178495 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425f601-c417-4441-b4b5-58d491588248 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:31.233019 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:31.294115 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425f601-c417-4441-b4b5-58d491588248 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:31.329320 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28203f-a6c3-46d5-b65e-cae556fc49f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:31.329995 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28203f-a6c3-46d5-b65e-cae556fc49f5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:31.330271 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28203f-a6c3-46d5-b65e-cae556fc49f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:31.330550 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28203f-a6c3-46d5-b65e-cae556fc49f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:31.331384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28203f-a6c3-46d5-b65e-cae556fc49f5 None admin] RBAC: Adding query filter params (role.id=6134821b0f644a8ea7655b40f2ab3cc3, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=8d7fe01ef64f46f6a0e77d5945219b33) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:31.331728 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28203f-a6c3-46d5-b65e-cae556fc49f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:31.333184 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28203f-a6c3-46d5-b65e-cae556fc49f5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:31.347647 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 531/1041] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:04:31 2022] GET /identity/v3/role_assignments?user.id=8d7fe01ef64f46f6a0e77d5945219b33&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=6134821b0f644a8ea7655b40f2ab3cc3 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:32.827263 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a3f6f-8985-429b-b489-8e217b23a306 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:32.827819 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a3f6f-8985-429b-b489-8e217b23a306 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:32.827819 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a3f6f-8985-429b-b489-8e217b23a306 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:32.829298 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 511/1042] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:32.835099 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14514b6e-fbc8-4cc5-b990-b76a884c2436 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:32.835564 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14514b6e-fbc8-4cc5-b990-b76a884c2436 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:32.835674 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14514b6e-fbc8-4cc5-b990-b76a884c2436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:32.855294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-14514b6e-fbc8-4cc5-b990-b76a884c2436 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:32.857386 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-14514b6e-fbc8-4cc5-b990-b76a884c2436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:32.927219 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 532/1043] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:32.951329 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a3f6f-8985-429b-b489-8e217b23a306 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:32.970047 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f07623-4d0d-412b-a5de-45a085959bb8 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:32.970348 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f07623-4d0d-412b-a5de-45a085959bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:32.970608 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f07623-4d0d-412b-a5de-45a085959bb8 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:32.972174 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 533/1044] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:04:32.979649 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a2f7d-1e21-400f-8ce8-dbaf4ae08ac8 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:32.979939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a2f7d-1e21-400f-8ce8-dbaf4ae08ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:32.980374 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a2f7d-1e21-400f-8ce8-dbaf4ae08ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:32.995004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:32.997731 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d95a2f7d-1e21-400f-8ce8-dbaf4ae08ac8 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:32.998864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d95a2f7d-1e21-400f-8ce8-dbaf4ae08ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:33.039918 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:33.066716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a3f6f-8985-429b-b489-8e217b23a306 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:33.079046 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 534/1045] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:33.102007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a2f7d-1e21-400f-8ce8-dbaf4ae08ac8 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:33.102999 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff02fb8-703d-430f-ad09-8da9f01f4f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:33.103835 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff02fb8-703d-430f-ad09-8da9f01f4f8e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:33.104104 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff02fb8-703d-430f-ad09-8da9f01f4f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:33.104390 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff02fb8-703d-430f-ad09-8da9f01f4f8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:33.105569 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff02fb8-703d-430f-ad09-8da9f01f4f8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:33.107032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff02fb8-703d-430f-ad09-8da9f01f4f8e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:33.108860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 512/1046] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:33.114199 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff02fb8-703d-430f-ad09-8da9f01f4f8e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:33.206007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a2f7d-1e21-400f-8ce8-dbaf4ae08ac8 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:33.232812 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff02fb8-703d-430f-ad09-8da9f01f4f8e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:33.234428 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:33.242332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b335abf-1b79-4b62-9f6e-78aec85761fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:33.243363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b335abf-1b79-4b62-9f6e-78aec85761fe None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:33.243817 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b335abf-1b79-4b62-9f6e-78aec85761fe None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:33.244249 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b335abf-1b79-4b62-9f6e-78aec85761fe None admin] PATH_INFO: `/v3/services/image` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:33.245038 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b335abf-1b79-4b62-9f6e-78aec85761fe None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:33.246558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b335abf-1b79-4b62-9f6e-78aec85761fe None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:33.261125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-7b335abf-1b79-4b62-9f6e-78aec85761fe None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 21:04:33.261125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:33.261125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:33.261125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:33.261125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:33.261125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:33.261125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:33.261125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:33.261125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 21:04:33.261125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:33.261125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:33.261125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:33.261125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:33.261125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:33.261125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:33.261125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:33.261125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:33.262330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:33.262330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:33.262330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:33.262330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:33.262330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:33.262330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:33.262330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:33.262330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:33.262330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:33.262330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:33.262330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:33.262330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:33.262330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:33.262330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:33.262330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:33.262330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:33.263379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:33.263379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:33.263379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:33.263379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:33.263379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:33.263379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:33.263379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:33.263379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:33.263379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:33.263379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:33.263379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:33.263379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:33.263379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 21:04:33.263379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:33.267650 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 535/1047] 10.222.0.27 () {56 vars in 1105 bytes} [Mon Jun 27 21:04:33 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:33.271351 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b335abf-1b79-4b62-9f6e-78aec85761fe None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:33.275942 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7928b8-ef56-46fb-81fc-8beeb6c6a6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:33.276640 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7928b8-ef56-46fb-81fc-8beeb6c6a6bd None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:33.276923 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7928b8-ef56-46fb-81fc-8beeb6c6a6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:33.277197 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7928b8-ef56-46fb-81fc-8beeb6c6a6bd None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:33.277913 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7928b8-ef56-46fb-81fc-8beeb6c6a6bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:33.279351 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7928b8-ef56-46fb-81fc-8beeb6c6a6bd None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:33.311640 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 513/1048] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Jun 27 21:04:33 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:04:33.314634 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:33.351278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f80d8-3ac8-4191-a59a-19b1ca7f20a0 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:33.351512 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f80d8-3ac8-4191-a59a-19b1ca7f20a0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:33.351786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f80d8-3ac8-4191-a59a-19b1ca7f20a0 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:33.353183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 514/1049] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:04:33.358904 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1a319-bf1a-4ff5-919f-e4f69bcbf861 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:33.359252 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1a319-bf1a-4ff5-919f-e4f69bcbf861 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:33.359538 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1a319-bf1a-4ff5-919f-e4f69bcbf861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:33.378009 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b3c1a319-bf1a-4ff5-919f-e4f69bcbf861 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:33.379261 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b3c1a319-bf1a-4ff5-919f-e4f69bcbf861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:33.383504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b335abf-1b79-4b62-9f6e-78aec85761fe None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:33.420330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b00fe-cf4f-427b-8b5e-61176af5afd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:33.421021 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b00fe-cf4f-427b-8b5e-61176af5afd0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:33.421303 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b00fe-cf4f-427b-8b5e-61176af5afd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:33.421578 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b00fe-cf4f-427b-8b5e-61176af5afd0 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:33.422380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b00fe-cf4f-427b-8b5e-61176af5afd0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:33.422693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b00fe-cf4f-427b-8b5e-61176af5afd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:33.424148 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b00fe-cf4f-427b-8b5e-61176af5afd0 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:33.434869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 536/1050] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:33.436197 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b00fe-cf4f-427b-8b5e-61176af5afd0 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:33.464995 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 515/1051] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:33.489149 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1a319-bf1a-4ff5-919f-e4f69bcbf861 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:33.548183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:33.551941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b00fe-cf4f-427b-8b5e-61176af5afd0 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:33.587354 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:33.590062 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c207f-e4fc-4508-9075-3a1bb42b8236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:33.590922 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c207f-e4fc-4508-9075-3a1bb42b8236 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:33.591552 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c207f-e4fc-4508-9075-3a1bb42b8236 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:33.591868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c207f-e4fc-4508-9075-3a1bb42b8236 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:33.592737 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c207f-e4fc-4508-9075-3a1bb42b8236 None admin] RBAC: Adding query filter params (type=image) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:33.593159 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c207f-e4fc-4508-9075-3a1bb42b8236 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:33.594825 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c207f-e4fc-4508-9075-3a1bb42b8236 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:33.600364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 537/1052] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:33.601864 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1a319-bf1a-4ff5-919f-e4f69bcbf861 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:33.605884 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c207f-e4fc-4508-9075-3a1bb42b8236 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:33.639101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35048d78-ff60-47f0-bd90-431be7c77605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:33.639961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35048d78-ff60-47f0-bd90-431be7c77605 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:33.640243 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35048d78-ff60-47f0-bd90-431be7c77605 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:33.640576 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35048d78-ff60-47f0-bd90-431be7c77605 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:33.653918 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35048d78-ff60-47f0-bd90-431be7c77605 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:33.662720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-35048d78-ff60-47f0-bd90-431be7c77605 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:04:33.662720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:33.662720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:33.662720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:33.662720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:33.662720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:33.662720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:33.662720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:33.662720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:33.662720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:33.662720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:33.662720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:33.662720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:33.662720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:33.662720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:33.662720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:33.663765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:33.663765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:33.663765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:33.663765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:33.663765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:33.663765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:33.663765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:33.663765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:33.663765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:33.663765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:33.663765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:33.663765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:33.663765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:33.663765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:33.663765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:33.663765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:33.664615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:33.664615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:33.664615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:33.664615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:33.664615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:33.664615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:33.664615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:33.664615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:33.664615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:33.664615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:33.664615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:04:33.664615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:33.664615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 516/1053] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Jun 27 21:04:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:33.670032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35048d78-ff60-47f0-bd90-431be7c77605 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:33.735234 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c207f-e4fc-4508-9075-3a1bb42b8236 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:33.770814 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fdc91-c482-4a9f-829a-6455b140266f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:33.771482 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fdc91-c482-4a9f-829a-6455b140266f None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:33.771802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fdc91-c482-4a9f-829a-6455b140266f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:33.772077 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fdc91-c482-4a9f-829a-6455b140266f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:33.772822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fdc91-c482-4a9f-829a-6455b140266f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:33.774201 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fdc91-c482-4a9f-829a-6455b140266f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:33.794882 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35048d78-ff60-47f0-bd90-431be7c77605 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:33.795091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 538/1054] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Jun 27 21:04:33 2022] POST /identity/v3/endpoints => generated 327 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:04:33.841258 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd3a27-af4b-474a-9841-b49b526e9628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:33.841982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd3a27-af4b-474a-9841-b49b526e9628 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:33.842225 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd3a27-af4b-474a-9841-b49b526e9628 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:33.842502 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd3a27-af4b-474a-9841-b49b526e9628 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:33.843258 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd3a27-af4b-474a-9841-b49b526e9628 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:33.843636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd3a27-af4b-474a-9841-b49b526e9628 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:33.845051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd3a27-af4b-474a-9841-b49b526e9628 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:33.860396 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 517/1055] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Jun 27 21:04:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:33.861896 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fdc91-c482-4a9f-829a-6455b140266f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:33.888401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:33.979809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fdc91-c482-4a9f-829a-6455b140266f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:34.019579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063c5b7-f6f2-438a-8a9b-a219955df0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:34.020726 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063c5b7-f6f2-438a-8a9b-a219955df0ea None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:34.021024 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063c5b7-f6f2-438a-8a9b-a219955df0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:34.021495 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063c5b7-f6f2-438a-8a9b-a219955df0ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:34.034127 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063c5b7-f6f2-438a-8a9b-a219955df0ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:34.035907 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063c5b7-f6f2-438a-8a9b-a219955df0ea None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:34.053889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-b063c5b7-f6f2-438a-8a9b-a219955df0ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:34.053889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:34.053889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:34.053889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:34.053889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:34.053889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:34.053889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:34.053889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:34.053889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:34.053889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:34.053889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:34.053889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:34.053889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:34.053889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:34.053889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:34.054794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:34.054794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:34.054794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:34.054794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:34.054794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:34.054794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:34.054794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:34.054794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:34.054794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:34.054794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:34.054794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:34.054794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:34.054794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:34.054794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:34.054794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:34.054794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:34.054794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:34.055765 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:34.055765 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:34.055765 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:34.055765 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:34.055765 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 539/1056] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:34.060786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd3a27-af4b-474a-9841-b49b526e9628 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:34.070161 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:34.168409 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd3a27-af4b-474a-9841-b49b526e9628 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:34.202863 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b64ad-559b-4995-923b-2c6e0997d6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:34.203620 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b64ad-559b-4995-923b-2c6e0997d6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:34.203922 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b64ad-559b-4995-923b-2c6e0997d6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:34.204195 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b64ad-559b-4995-923b-2c6e0997d6a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:34.204971 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b64ad-559b-4995-923b-2c6e0997d6a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:34.205278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b64ad-559b-4995-923b-2c6e0997d6a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:34.206730 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b64ad-559b-4995-923b-2c6e0997d6a9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:34.222563 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 518/1057] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:34.223891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063c5b7-f6f2-438a-8a9b-a219955df0ea None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:34.304205 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:34.336830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063c5b7-f6f2-438a-8a9b-a219955df0ea None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:34.370458 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d611dc-c798-4f90-83d0-c1832cd9fc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:34.371264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d611dc-c798-4f90-83d0-c1832cd9fc38 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:34.371547 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d611dc-c798-4f90-83d0-c1832cd9fc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:34.371833 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d611dc-c798-4f90-83d0-c1832cd9fc38 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:34.379531 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d611dc-c798-4f90-83d0-c1832cd9fc38 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:34.381027 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d611dc-c798-4f90-83d0-c1832cd9fc38 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:34.401143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-d3d611dc-c798-4f90-83d0-c1832cd9fc38 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:04:34.401143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:34.401143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:34.401143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:34.401143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:34.401143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:34.401143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:34.401143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:34.401143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:34.401143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:34.401143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:34.401143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:34.401143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:04:34.401143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:04:34.401143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:04:34.401143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:04:34.402111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:34.402111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:34.402111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:04:34.402111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:34.402111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:04:34.402111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:34.402111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:34.402111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:34.402111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:34.402111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:34.402111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:34.402111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:34.402111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:34.402111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:34.402111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:34.402111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:34.402111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:34.402111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:34.403060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:34.403060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:34.403060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:04:34.403060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:04:34.403060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:04:34.403060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:04:34.403060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:04:34.403060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:04:34.403060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:04:34.403060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:34.403060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 540/1058] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Jun 27 21:04:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:34.409868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b64ad-559b-4995-923b-2c6e0997d6a9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:34.485376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:34.522001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b64ad-559b-4995-923b-2c6e0997d6a9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:34.556803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231a80b-fa70-4860-87d6-e6e51892f84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:34.557773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231a80b-fa70-4860-87d6-e6e51892f84d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:34.558256 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231a80b-fa70-4860-87d6-e6e51892f84d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:34.558702 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231a80b-fa70-4860-87d6-e6e51892f84d None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:34.559741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231a80b-fa70-4860-87d6-e6e51892f84d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:34.560222 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231a80b-fa70-4860-87d6-e6e51892f84d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:34.561824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231a80b-fa70-4860-87d6-e6e51892f84d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:34.579798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 519/1059] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Jun 27 21:04:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:34.585481 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d611dc-c798-4f90-83d0-c1832cd9fc38 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:34.686758 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d611dc-c798-4f90-83d0-c1832cd9fc38 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:34.715999 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:34.722120 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e042e-3801-416a-84b5-0e6d181d67cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:34.722927 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e042e-3801-416a-84b5-0e6d181d67cb None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:34.723208 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e042e-3801-416a-84b5-0e6d181d67cb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:34.723481 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e042e-3801-416a-84b5-0e6d181d67cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:34.734224 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e042e-3801-416a-84b5-0e6d181d67cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:34.735781 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e042e-3801-416a-84b5-0e6d181d67cb None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:34.752735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-3d6e042e-3801-416a-84b5-0e6d181d67cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:34.752735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:34.752735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:34.752735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:34.752735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:34.752735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:34.752735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:34.752735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:34.752735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:34.752735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:34.752735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:34.752735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:34.752735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:34.752735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:34.752735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:34.753583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:34.753583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:34.753583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:34.753583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:34.753583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:34.753583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:34.753583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:34.753583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:34.753583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:34.753583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:34.753583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:34.753583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:34.753583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:34.753583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:34.753583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:34.753583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:34.753583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:34.754921 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:34.754921 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:34.754921 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:34.754921 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:34.754921 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 541/1060] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:34.759664 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231a80b-fa70-4860-87d6-e6e51892f84d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:34.860926 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231a80b-fa70-4860-87d6-e6e51892f84d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:34.885321 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:34.896879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2853a54-25bb-404c-b240-0bf82d164034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:34.897558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2853a54-25bb-404c-b240-0bf82d164034 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:34.897847 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2853a54-25bb-404c-b240-0bf82d164034 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:34.898124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2853a54-25bb-404c-b240-0bf82d164034 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:34.898848 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2853a54-25bb-404c-b240-0bf82d164034 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:34.899157 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2853a54-25bb-404c-b240-0bf82d164034 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:34.900560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2853a54-25bb-404c-b240-0bf82d164034 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:34.912904 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 520/1061] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:34.918380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e042e-3801-416a-84b5-0e6d181d67cb None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:35.027870 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e042e-3801-416a-84b5-0e6d181d67cb None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:35.063567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32af4fc-7e58-47bf-b9d4-0ef151982265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:35.064414 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32af4fc-7e58-47bf-b9d4-0ef151982265 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:35.064699 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32af4fc-7e58-47bf-b9d4-0ef151982265 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:35.064946 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32af4fc-7e58-47bf-b9d4-0ef151982265 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:35.072189 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32af4fc-7e58-47bf-b9d4-0ef151982265 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:35.073686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32af4fc-7e58-47bf-b9d4-0ef151982265 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:35.091483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-d32af4fc-7e58-47bf-b9d4-0ef151982265 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:35.091483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:35.091483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:35.091483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:35.091483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:35.091483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:35.091483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:35.091483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:35.091483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:35.091483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:35.091483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:35.091483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:35.091483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:04:35.091483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:04:35.091483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:04:35.092557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:04:35.092557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:35.092557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:35.092557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:35.092557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:35.092557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:35.092557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:35.092557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:35.092557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:35.092557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:35.092557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:35.092557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:35.092557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:35.092557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:35.092557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:35.092557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:35.092557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:35.093459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:04:35.093459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:04:35.093459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:04:35.093459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:04:35.093459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:04:35.093459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:04:35.093459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:35.093459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:35.093459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 542/1062] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:04:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:35.100182 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2853a54-25bb-404c-b240-0bf82d164034 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:35.205887 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2853a54-25bb-404c-b240-0bf82d164034 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:35.240911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c52ac3-7c46-4747-b18d-bdd6ea27a937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:35.241582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c52ac3-7c46-4747-b18d-bdd6ea27a937 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:35.241863 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c52ac3-7c46-4747-b18d-bdd6ea27a937 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:35.242136 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c52ac3-7c46-4747-b18d-bdd6ea27a937 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:35.242926 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c52ac3-7c46-4747-b18d-bdd6ea27a937 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:35.243241 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c52ac3-7c46-4747-b18d-bdd6ea27a937 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:35.244717 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c52ac3-7c46-4747-b18d-bdd6ea27a937 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:35.262116 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 521/1063] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:04:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:35.263577 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32af4fc-7e58-47bf-b9d4-0ef151982265 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:35.312261 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:35.369436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32af4fc-7e58-47bf-b9d4-0ef151982265 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:35.404258 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d3b46-b740-4707-a1f3-5c765e75176c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:35.405156 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d3b46-b740-4707-a1f3-5c765e75176c None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:35.405577 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d3b46-b740-4707-a1f3-5c765e75176c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:35.405720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d3b46-b740-4707-a1f3-5c765e75176c None admin] PATH_INFO: `/v3/projects/dfc29b021d1646819137c89e93f33601/users/8d7fe01ef64f46f6a0e77d5945219b33/roles/6134821b0f644a8ea7655b40f2ab3cc3` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:35.407652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ea746-7508-441b-b525-ed798692c560 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:35.407960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ea746-7508-441b-b525-ed798692c560 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:35.408231 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ea746-7508-441b-b525-ed798692c560 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:35.408497 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d3b46-b740-4707-a1f3-5c765e75176c None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc29b021d1646819137c89e93f33601, user_id=8d7fe01ef64f46f6a0e77d5945219b33, role_id=6134821b0f644a8ea7655b40f2ab3cc3)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:35.409755 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 522/1064] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:04:35.409922 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d3b46-b740-4707-a1f3-5c765e75176c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:35.414586 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437cdc9-8cbb-44ea-8d7d-9a950f5a48f9 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:35.414858 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437cdc9-8cbb-44ea-8d7d-9a950f5a48f9 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:35.415394 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437cdc9-8cbb-44ea-8d7d-9a950f5a48f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:35.431640 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 543/1065] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:04:35 2022] PUT /identity/v3/projects/dfc29b021d1646819137c89e93f33601/users/8d7fe01ef64f46f6a0e77d5945219b33/roles/6134821b0f644a8ea7655b40f2ab3cc3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:04:35.438914 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5437cdc9-8cbb-44ea-8d7d-9a950f5a48f9 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:35.440304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5437cdc9-8cbb-44ea-8d7d-9a950f5a48f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:35.488049 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:35.523649 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 523/1066] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:35.545891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d3b46-b740-4707-a1f3-5c765e75176c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:35.654265 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d3b46-b740-4707-a1f3-5c765e75176c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:35.658342 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:35.692520 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d576f7-d98c-4563-a9b5-6586506a2177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:35.693686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d576f7-d98c-4563-a9b5-6586506a2177 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:35.694184 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d576f7-d98c-4563-a9b5-6586506a2177 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:35.694666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d576f7-d98c-4563-a9b5-6586506a2177 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:35.706428 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d576f7-d98c-4563-a9b5-6586506a2177 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:35.714512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-52d576f7-d98c-4563-a9b5-6586506a2177 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:35.714512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:35.714512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:35.714512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:35.714512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:35.714512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:35.714512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:35.714512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:35.714512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:35.714512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:35.714512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:35.714512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:35.714512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:35.714512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:35.714512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:35.714512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:35.715444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:35.715444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:35.715444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:35.715444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:35.715444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:35.715444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:35.715444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:35.715444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:35.715444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:35.715444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:35.715444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:35.715444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:35.715444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:35.715444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:35.715444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:35.715444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:35.716312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:35.716312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:35.716312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:35.716312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:35.716312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:35.716312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:35.716312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:35.716312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:35.716312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:35.716312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:35.716312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:35.716312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:35.723676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 544/1067] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:04:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:35.725548 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437cdc9-8cbb-44ea-8d7d-9a950f5a48f9 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:35.831641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437cdc9-8cbb-44ea-8d7d-9a950f5a48f9 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:35.857122 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f227a-a487-4f55-9d23-713b684c2844 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:35.857837 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f227a-a487-4f55-9d23-713b684c2844 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:35.858125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f227a-a487-4f55-9d23-713b684c2844 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:35.860093 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 545/1068] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:35.865794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f0f72-2238-46be-af59-6dfd34e7976c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:35.866178 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f0f72-2238-46be-af59-6dfd34e7976c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:35.866461 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f0f72-2238-46be-af59-6dfd34e7976c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:35.872217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47dc58-a3f9-4460-ba52-0ce5a4f9553b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:35.873011 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47dc58-a3f9-4460-ba52-0ce5a4f9553b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:35.873291 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47dc58-a3f9-4460-ba52-0ce5a4f9553b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:35.873564 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47dc58-a3f9-4460-ba52-0ce5a4f9553b None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:35.874316 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47dc58-a3f9-4460-ba52-0ce5a4f9553b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:35.874619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47dc58-a3f9-4460-ba52-0ce5a4f9553b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:35.876106 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47dc58-a3f9-4460-ba52-0ce5a4f9553b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:35.884963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2e5f0f72-2238-46be-af59-6dfd34e7976c None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:35.886180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2e5f0f72-2238-46be-af59-6dfd34e7976c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:35.887674 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 524/1069] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:04:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:35.891552 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47dc58-a3f9-4460-ba52-0ce5a4f9553b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:35.929281 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:35.959675 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 546/1070] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:35.981675 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f0f72-2238-46be-af59-6dfd34e7976c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:36.009330 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47dc58-a3f9-4460-ba52-0ce5a4f9553b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:36.045045 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691a531-0c1b-473d-87fc-3f6a632a15d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:36.046262 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691a531-0c1b-473d-87fc-3f6a632a15d5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:36.046643 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691a531-0c1b-473d-87fc-3f6a632a15d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:36.046944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691a531-0c1b-473d-87fc-3f6a632a15d5 None admin] PATH_INFO: `/v3/users/307303ec9d4244c398b884e73ff364a3` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:36.064268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691a531-0c1b-473d-87fc-3f6a632a15d5 None admin] RBAC: Authorizing `identity:get_user(user_id=307303ec9d4244c398b884e73ff364a3)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:36.065743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691a531-0c1b-473d-87fc-3f6a632a15d5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:36.067790 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 525/1071] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:35 2022] GET /identity/v3/users/307303ec9d4244c398b884e73ff364a3 => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:36.073475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691a531-0c1b-473d-87fc-3f6a632a15d5 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:36.087478 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:36.095323 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f0f72-2238-46be-af59-6dfd34e7976c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:36.131152 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d7246-68e8-4981-8399-5d4931a99d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:36.131854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d7246-68e8-4981-8399-5d4931a99d38 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:36.132139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d7246-68e8-4981-8399-5d4931a99d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:36.132417 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d7246-68e8-4981-8399-5d4931a99d38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:36.173899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:36.182173 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691a531-0c1b-473d-87fc-3f6a632a15d5 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:36.203275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d7246-68e8-4981-8399-5d4931a99d38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:36.204619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d7246-68e8-4981-8399-5d4931a99d38 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:36.217788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d630e6a-6b81-4118-a91e-7f42693b4cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:36.218524 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d630e6a-6b81-4118-a91e-7f42693b4cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:36.218826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d630e6a-6b81-4118-a91e-7f42693b4cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:36.219088 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d630e6a-6b81-4118-a91e-7f42693b4cb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:36.293223 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d630e6a-6b81-4118-a91e-7f42693b4cb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:36.294698 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:36.295715 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d630e6a-6b81-4118-a91e-7f42693b4cb6 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:36.303537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 547/1072] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 21:04:36.309284 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d7246-68e8-4981-8399-5d4931a99d38 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:36.387515 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:36.396110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 526/1073] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:36.401544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d630e6a-6b81-4118-a91e-7f42693b4cb6 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:36.420548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d7246-68e8-4981-8399-5d4931a99d38 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:36.457717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d534f-807e-4cb8-8d80-4b539cc2ca14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:36.458528 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d534f-807e-4cb8-8d80-4b539cc2ca14 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:36.458809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d534f-807e-4cb8-8d80-4b539cc2ca14 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:36.459086 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d534f-807e-4cb8-8d80-4b539cc2ca14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:36.470370 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d534f-807e-4cb8-8d80-4b539cc2ca14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:36.471916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d534f-807e-4cb8-8d80-4b539cc2ca14 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:36.492555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f68d534f-807e-4cb8-8d80-4b539cc2ca14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:36.492555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:36.492555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:36.492555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:36.492555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:36.492555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:36.492555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:36.492555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:36.492555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:36.492555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:36.492555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:36.492555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:36.492555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:36.492555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:36.492555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:36.493506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:36.493506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:36.493506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:36.493506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:36.493506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:36.493506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:36.493506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:36.493506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:36.493506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:36.493506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:36.493506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:36.493506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:36.493506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:36.493506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:36.493506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:36.493506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:36.493506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:36.494407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:36.494407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:36.494407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:36.494407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:36.495211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 548/1074] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:36.499847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d534f-807e-4cb8-8d80-4b539cc2ca14 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:36.519250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d630e6a-6b81-4118-a91e-7f42693b4cb6 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:36.550587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:36.557850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20aa375-723a-4123-87f8-45494721f086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:36.558808 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20aa375-723a-4123-87f8-45494721f086 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:36.559097 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20aa375-723a-4123-87f8-45494721f086 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:36.559372 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20aa375-723a-4123-87f8-45494721f086 None admin] PATH_INFO: `/v3/projects/2ef50902b46a4e29856a2bedb211310b` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:36.560585 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20aa375-723a-4123-87f8-45494721f086 None admin] RBAC: Authorizing `identity:get_project(project_id=2ef50902b46a4e29856a2bedb211310b)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:36.562027 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20aa375-723a-4123-87f8-45494721f086 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:36.563794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 527/1075] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:04:36 2022] GET /identity/v3/projects/2ef50902b46a4e29856a2bedb211310b => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:36.569647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20aa375-723a-4123-87f8-45494721f086 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:36.605792 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d534f-807e-4cb8-8d80-4b539cc2ca14 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:36.639381 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:36.642794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a6e02-d9dd-406c-9726-8cb2257fe297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:36.643495 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a6e02-d9dd-406c-9726-8cb2257fe297 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:36.644086 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a6e02-d9dd-406c-9726-8cb2257fe297 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:36.644478 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a6e02-d9dd-406c-9726-8cb2257fe297 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:36.645147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a6e02-d9dd-406c-9726-8cb2257fe297 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:36.645426 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a6e02-d9dd-406c-9726-8cb2257fe297 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:36.646823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a6e02-d9dd-406c-9726-8cb2257fe297 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:36.659415 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 549/1076] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:36.688086 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20aa375-723a-4123-87f8-45494721f086 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:36.729606 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9cafae-2c57-4994-ad2d-99159effd7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:36.730293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9cafae-2c57-4994-ad2d-99159effd7af None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:36.730577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9cafae-2c57-4994-ad2d-99159effd7af None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:36.730854 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9cafae-2c57-4994-ad2d-99159effd7af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:36.731748 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9cafae-2c57-4994-ad2d-99159effd7af None admin] RBAC: Adding query filter params (role.id=5c3c095bb53943eca95c4e049cd8cbf5, scope.project.id=2ef50902b46a4e29856a2bedb211310b, user.id=307303ec9d4244c398b884e73ff364a3) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:36.732075 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9cafae-2c57-4994-ad2d-99159effd7af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:36.733547 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9cafae-2c57-4994-ad2d-99159effd7af None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:36.740560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 528/1077] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:04:36 2022] GET /identity/v3/role_assignments?user.id=307303ec9d4244c398b884e73ff364a3&scope.project.id=2ef50902b46a4e29856a2bedb211310b&role.id=5c3c095bb53943eca95c4e049cd8cbf5 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:37.424652 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ce7b1-c7e9-444a-8a8f-2a0b8e1f3ac1 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:37.425210 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ce7b1-c7e9-444a-8a8f-2a0b8e1f3ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:37.425210 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ce7b1-c7e9-444a-8a8f-2a0b8e1f3ac1 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:37.426579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 550/1078] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:37.431981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17682f-f26e-479a-8856-f66e3cbb5fda None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:37.432428 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17682f-f26e-479a-8856-f66e3cbb5fda None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:37.432521 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17682f-f26e-479a-8856-f66e3cbb5fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:37.449776 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ab17682f-f26e-479a-8856-f66e3cbb5fda None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:37.450964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ab17682f-f26e-479a-8856-f66e3cbb5fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:37.520438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 529/1079] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:37.544267 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ce7b1-c7e9-444a-8a8f-2a0b8e1f3ac1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:37.654441 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ce7b1-c7e9-444a-8a8f-2a0b8e1f3ac1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:37.662100 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:37.691518 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c38b2b-0dc1-4779-9c83-ba3d866720df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:37.692291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c38b2b-0dc1-4779-9c83-ba3d866720df None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:37.692574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c38b2b-0dc1-4779-9c83-ba3d866720df None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:37.692844 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c38b2b-0dc1-4779-9c83-ba3d866720df None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:37.703645 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c38b2b-0dc1-4779-9c83-ba3d866720df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:37.712023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-75c38b2b-0dc1-4779-9c83-ba3d866720df None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:04:37.712023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:37.712023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:37.712023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:37.712023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:37.712023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:37.712023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:37.712023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:37.712023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:37.712023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:37.712023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:37.712023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:37.712023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:37.712023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:37.712023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:37.712023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:37.713054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:37.713054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:37.713054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:37.713054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:37.713054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:37.713054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:37.713054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:37.713054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:37.713054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:37.713054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:37.713054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:37.713054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:37.713054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:37.713054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:37.713054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:37.713054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:37.713928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:37.713928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:37.713928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:37.713928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:37.713928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:37.713928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:37.713928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:37.713928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:37.713928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:37.713928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:37.713928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:04:37.713928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:37.713928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 551/1080] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Jun 27 21:04:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:37.719441 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17682f-f26e-479a-8856-f66e3cbb5fda None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:37.778802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:37.826788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17682f-f26e-479a-8856-f66e3cbb5fda None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:37.864097 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838cdbd-9973-484b-b740-d42054e55459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:37.864778 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838cdbd-9973-484b-b740-d42054e55459 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:37.865061 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838cdbd-9973-484b-b740-d42054e55459 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:37.865320 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838cdbd-9973-484b-b740-d42054e55459 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:37.866068 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838cdbd-9973-484b-b740-d42054e55459 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:37.866374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838cdbd-9973-484b-b740-d42054e55459 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:37.867784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838cdbd-9973-484b-b740-d42054e55459 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:37.876022 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 530/1081] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Jun 27 21:04:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:37.881796 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c38b2b-0dc1-4779-9c83-ba3d866720df None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:37.991459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c38b2b-0dc1-4779-9c83-ba3d866720df None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:38.032662 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b71ac-e302-485f-9830-1ba260989d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:38.033473 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b71ac-e302-485f-9830-1ba260989d4f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:38.033757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b71ac-e302-485f-9830-1ba260989d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:38.034031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b71ac-e302-485f-9830-1ba260989d4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:38.045021 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b71ac-e302-485f-9830-1ba260989d4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:38.046513 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b71ac-e302-485f-9830-1ba260989d4f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:38.063868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4d5b71ac-e302-485f-9830-1ba260989d4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:38.063868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:38.063868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:38.063868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:38.063868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:38.063868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:38.063868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:38.063868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:38.063868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:38.063868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:38.063868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:38.063868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:38.063868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:38.063868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:38.063868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:38.064771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:38.064771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:38.064771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:38.064771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:38.064771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:38.064771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:38.064771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:38.064771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:38.064771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:38.064771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:38.064771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:38.064771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:38.064771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:38.064771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:38.064771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:38.064771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:38.064771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:38.065830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:38.065830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:38.065830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:38.065830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:38.065830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 552/1082] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:38.070965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838cdbd-9973-484b-b740-d42054e55459 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:38.177737 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838cdbd-9973-484b-b740-d42054e55459 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:38.195468 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:38.213539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6d3f1-c414-42b7-bdf8-1ad53d27131c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:38.214201 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6d3f1-c414-42b7-bdf8-1ad53d27131c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:38.214484 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6d3f1-c414-42b7-bdf8-1ad53d27131c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:38.214767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6d3f1-c414-42b7-bdf8-1ad53d27131c None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:38.215498 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6d3f1-c414-42b7-bdf8-1ad53d27131c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:38.215840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6d3f1-c414-42b7-bdf8-1ad53d27131c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:38.217235 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6d3f1-c414-42b7-bdf8-1ad53d27131c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:38.230007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 531/1083] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:38.236083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b71ac-e302-485f-9830-1ba260989d4f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:38.256934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:38.340483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b71ac-e302-485f-9830-1ba260989d4f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:38.375526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab66a0-6f27-4890-8ce0-36a9299d801e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:38.376365 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab66a0-6f27-4890-8ce0-36a9299d801e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:38.376652 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab66a0-6f27-4890-8ce0-36a9299d801e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:38.376928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab66a0-6f27-4890-8ce0-36a9299d801e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:38.384761 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab66a0-6f27-4890-8ce0-36a9299d801e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:38.386229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab66a0-6f27-4890-8ce0-36a9299d801e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:38.400616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-20ab66a0-6f27-4890-8ce0-36a9299d801e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:04:38.400616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:38.400616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:38.400616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:38.400616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:38.400616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:38.400616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:38.400616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:38.400616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:38.400616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:38.400616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:38.400616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:38.400616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:04:38.400616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:04:38.400616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:04:38.400616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:04:38.401579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:38.401579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:38.401579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:04:38.401579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:38.401579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:04:38.401579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:38.401579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:38.401579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:38.401579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:38.401579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:38.401579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:38.401579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:38.401579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:38.401579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:38.401579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:38.401579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:38.401579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:38.401579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:38.402878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:38.402878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:38.402878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:04:38.402878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:04:38.402878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:04:38.402878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:04:38.402878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:04:38.402878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:04:38.402878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:04:38.402878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:38.402878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 553/1084] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Jun 27 21:04:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:38.408316 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6d3f1-c414-42b7-bdf8-1ad53d27131c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:38.514246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6d3f1-c414-42b7-bdf8-1ad53d27131c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:38.549824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa09ce-f9ab-447b-89d9-9c574a06c498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:38.550501 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa09ce-f9ab-447b-89d9-9c574a06c498 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:38.550784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa09ce-f9ab-447b-89d9-9c574a06c498 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:38.551053 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa09ce-f9ab-447b-89d9-9c574a06c498 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:38.551907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa09ce-f9ab-447b-89d9-9c574a06c498 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:38.552220 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa09ce-f9ab-447b-89d9-9c574a06c498 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:38.553665 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa09ce-f9ab-447b-89d9-9c574a06c498 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:38.571085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 532/1085] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Jun 27 21:04:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:38.577030 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab66a0-6f27-4890-8ce0-36a9299d801e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:38.661075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:38.683920 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab66a0-6f27-4890-8ce0-36a9299d801e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:38.718410 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6970dff-206e-42e1-bb40-526175177bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:38.719077 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6970dff-206e-42e1-bb40-526175177bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:38.719360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6970dff-206e-42e1-bb40-526175177bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:38.719658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6970dff-206e-42e1-bb40-526175177bf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:38.747257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a0dc1-6315-4a00-96e9-435125c1b013 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:38.747547 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a0dc1-6315-4a00-96e9-435125c1b013 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:38.747880 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a0dc1-6315-4a00-96e9-435125c1b013 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:38.749233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 533/1086] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:38.755094 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be0e73-60db-4347-89b5-ed5175634b8d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:38.755439 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be0e73-60db-4347-89b5-ed5175634b8d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:38.755803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be0e73-60db-4347-89b5-ed5175634b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:38.772447 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-54be0e73-60db-4347-89b5-ed5175634b8d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:38.773578 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-54be0e73-60db-4347-89b5-ed5175634b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:38.789823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6970dff-206e-42e1-bb40-526175177bf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:38.791300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6970dff-206e-42e1-bb40-526175177bf3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:38.804174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:38.844022 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 534/1087] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:38.866608 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be0e73-60db-4347-89b5-ed5175634b8d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:38.869714 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:38.890714 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 554/1088] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:38.896433 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6970dff-206e-42e1-bb40-526175177bf3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:38.965436 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be0e73-60db-4347-89b5-ed5175634b8d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:38.995899 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6970dff-206e-42e1-bb40-526175177bf3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:39.001664 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5610240-1476-4edc-997e-3f386226471f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:39.002485 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5610240-1476-4edc-997e-3f386226471f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:39.002774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5610240-1476-4edc-997e-3f386226471f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:39.003054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5610240-1476-4edc-997e-3f386226471f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:39.013399 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5610240-1476-4edc-997e-3f386226471f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:39.021366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-e5610240-1476-4edc-997e-3f386226471f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:39.021366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:39.021366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:39.021366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:39.021366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:39.021366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:39.021366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:39.021366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:39.021366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:39.021366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:39.021366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:39.021366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:39.021366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:39.021366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:39.021366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:39.021366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:39.022528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:39.022528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:39.022528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:39.022528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:39.022528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:39.022528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:39.022528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:39.022528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:39.022528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:39.022528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:39.022528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:39.022528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:39.022528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:39.022528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:39.022528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:39.022528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:39.023584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:39.023584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:39.023584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:39.023584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:39.023584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:39.023584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:39.023584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:39.023584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:39.023584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:39.023584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:39.023584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:39.023584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:39.023584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 535/1089] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:04:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:39.028987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5610240-1476-4edc-997e-3f386226471f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:39.033494 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2cfbd-0340-40b0-aa4d-bd3bce71623d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:39.034298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2cfbd-0340-40b0-aa4d-bd3bce71623d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:39.034576 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2cfbd-0340-40b0-aa4d-bd3bce71623d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:39.034854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2cfbd-0340-40b0-aa4d-bd3bce71623d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:39.045982 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf2cfbd-0340-40b0-aa4d-bd3bce71623d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:39.047567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf2cfbd-0340-40b0-aa4d-bd3bce71623d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:39.064869 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:39.065943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-9cf2cfbd-0340-40b0-aa4d-bd3bce71623d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:39.065943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:39.065943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:39.065943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:39.065943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:39.065943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:39.065943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:39.065943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:39.065943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:39.065943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:39.065943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:39.065943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:39.065943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:39.065943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:39.065943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:39.067098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:39.067098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:39.067098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:39.067098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:39.067098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:39.067098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:39.067098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:39.067098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:39.067098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:39.067098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:39.067098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:39.067098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:39.067098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:39.067098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:39.067098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:39.067098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:39.067098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:39.068525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:39.068525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:39.068525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:39.068525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:39.068525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 555/1090] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:39.072934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2cfbd-0340-40b0-aa4d-bd3bce71623d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:39.103694 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:39.141153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5610240-1476-4edc-997e-3f386226471f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:39.174692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070af0e6-bb47-4dce-920a-965e7002dfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:39.175396 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070af0e6-bb47-4dce-920a-965e7002dfd8 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:39.175709 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070af0e6-bb47-4dce-920a-965e7002dfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:39.175876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2cfbd-0340-40b0-aa4d-bd3bce71623d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:39.175972 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070af0e6-bb47-4dce-920a-965e7002dfd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:39.176720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070af0e6-bb47-4dce-920a-965e7002dfd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:39.176991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070af0e6-bb47-4dce-920a-965e7002dfd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:39.178409 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070af0e6-bb47-4dce-920a-965e7002dfd8 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:39.186745 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 536/1091] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:04:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:39.192438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070af0e6-bb47-4dce-920a-965e7002dfd8 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:39.209559 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19668762-8e89-401c-a516-43b72492409f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:39.210238 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19668762-8e89-401c-a516-43b72492409f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:39.210517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19668762-8e89-401c-a516-43b72492409f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:39.210790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19668762-8e89-401c-a516-43b72492409f None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:39.211515 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19668762-8e89-401c-a516-43b72492409f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:39.211861 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19668762-8e89-401c-a516-43b72492409f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:39.213241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19668762-8e89-401c-a516-43b72492409f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:39.224938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 556/1092] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:39.230478 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19668762-8e89-401c-a516-43b72492409f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:39.279751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:39.292761 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070af0e6-bb47-4dce-920a-965e7002dfd8 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:39.319131 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:39.329722 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580d878-abc4-495d-905a-7334ec8b0af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:39.329878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19668762-8e89-401c-a516-43b72492409f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:39.330612 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580d878-abc4-495d-905a-7334ec8b0af5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:39.330893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580d878-abc4-495d-905a-7334ec8b0af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:39.331167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580d878-abc4-495d-905a-7334ec8b0af5 None admin] PATH_INFO: `/v3/users/307303ec9d4244c398b884e73ff364a3` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:39.332404 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580d878-abc4-495d-905a-7334ec8b0af5 None admin] RBAC: Authorizing `identity:get_user(user_id=307303ec9d4244c398b884e73ff364a3)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:39.333852 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580d878-abc4-495d-905a-7334ec8b0af5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:39.335491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 537/1093] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:39 2022] GET /identity/v3/users/307303ec9d4244c398b884e73ff364a3 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:39.341548 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580d878-abc4-495d-905a-7334ec8b0af5 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:39.364963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf561e-0f6c-41a2-b99d-1a31a1907613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:39.365775 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf561e-0f6c-41a2-b99d-1a31a1907613 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:39.366054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf561e-0f6c-41a2-b99d-1a31a1907613 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:39.366326 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf561e-0f6c-41a2-b99d-1a31a1907613 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:39.373447 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf561e-0f6c-41a2-b99d-1a31a1907613 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:39.374979 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf561e-0f6c-41a2-b99d-1a31a1907613 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:39.389250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-83cf561e-0f6c-41a2-b99d-1a31a1907613 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:39.389250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:39.389250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:39.389250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:39.389250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:39.389250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:39.389250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:39.389250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:39.389250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:39.389250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:39.389250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:39.389250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:39.389250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:04:39.389250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:04:39.389250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:04:39.390246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:04:39.390246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:39.390246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:39.390246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:39.390246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:39.390246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:39.390246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:39.390246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:39.390246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:39.390246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:39.390246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:39.390246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:39.390246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:39.390246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:39.390246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:39.390246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:39.390246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:39.391123 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:04:39.391123 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:04:39.391123 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:04:39.391123 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:04:39.391123 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:04:39.391123 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:04:39.391123 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:39.391123 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:04:39.391123 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 557/1094] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:04:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:39.396241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf561e-0f6c-41a2-b99d-1a31a1907613 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:39.445144 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580d878-abc4-495d-905a-7334ec8b0af5 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:39.478435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548e8f9-78b6-4091-9120-b335b8b169e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:39.479305 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548e8f9-78b6-4091-9120-b335b8b169e3 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:39.479591 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548e8f9-78b6-4091-9120-b335b8b169e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:39.479893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548e8f9-78b6-4091-9120-b335b8b169e3 None admin] PATH_INFO: `/v3/projects/2ef50902b46a4e29856a2bedb211310b` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:39.481102 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548e8f9-78b6-4091-9120-b335b8b169e3 None admin] RBAC: Authorizing `identity:get_project(project_id=2ef50902b46a4e29856a2bedb211310b)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:39.482565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548e8f9-78b6-4091-9120-b335b8b169e3 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:39.484332 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 538/1095] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:04:39 2022] GET /identity/v3/projects/2ef50902b46a4e29856a2bedb211310b => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:39.490822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548e8f9-78b6-4091-9120-b335b8b169e3 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:39.493191 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf561e-0f6c-41a2-b99d-1a31a1907613 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:39.516104 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:39.527237 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11425df6-b0ce-4cc0-af5e-2f0719b22886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:39.527945 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11425df6-b0ce-4cc0-af5e-2f0719b22886 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:39.528229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11425df6-b0ce-4cc0-af5e-2f0719b22886 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:39.528504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11425df6-b0ce-4cc0-af5e-2f0719b22886 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:39.529293 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11425df6-b0ce-4cc0-af5e-2f0719b22886 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:39.529602 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11425df6-b0ce-4cc0-af5e-2f0719b22886 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:39.531138 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11425df6-b0ce-4cc0-af5e-2f0719b22886 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:39.543221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 558/1096] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:04:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:39.549191 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11425df6-b0ce-4cc0-af5e-2f0719b22886 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:39.561186 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:39.592912 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548e8f9-78b6-4091-9120-b335b8b169e3 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:39.627253 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890b47e-fab3-499f-b696-88f29afa9407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:39.628202 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890b47e-fab3-499f-b696-88f29afa9407 None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:39.628478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890b47e-fab3-499f-b696-88f29afa9407 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:39.628744 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890b47e-fab3-499f-b696-88f29afa9407 None admin] PATH_INFO: `/v3/projects/2ef50902b46a4e29856a2bedb211310b/users/307303ec9d4244c398b884e73ff364a3/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:39.630731 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890b47e-fab3-499f-b696-88f29afa9407 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ef50902b46a4e29856a2bedb211310b, user_id=307303ec9d4244c398b884e73ff364a3, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:39.632198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890b47e-fab3-499f-b696-88f29afa9407 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:39.646368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 539/1097] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:04:39 2022] PUT /identity/v3/projects/2ef50902b46a4e29856a2bedb211310b/users/307303ec9d4244c398b884e73ff364a3/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:04:39.651999 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11425df6-b0ce-4cc0-af5e-2f0719b22886 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:39.691249 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1461838-4a85-4108-8fb1-bf9ed6465577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:39.691979 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1461838-4a85-4108-8fb1-bf9ed6465577 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:39.692260 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1461838-4a85-4108-8fb1-bf9ed6465577 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:39.692537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1461838-4a85-4108-8fb1-bf9ed6465577 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:39.693391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1461838-4a85-4108-8fb1-bf9ed6465577 None admin] RBAC: Adding query filter params (role.id=6134821b0f644a8ea7655b40f2ab3cc3, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=8d7fe01ef64f46f6a0e77d5945219b33) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:39.693704 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1461838-4a85-4108-8fb1-bf9ed6465577 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:39.695197 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1461838-4a85-4108-8fb1-bf9ed6465577 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:39.703177 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 559/1098] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:04:39 2022] GET /identity/v3/role_assignments?user.id=8d7fe01ef64f46f6a0e77d5945219b33&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=6134821b0f644a8ea7655b40f2ab3cc3 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:41.659309 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b40b3d-6f5c-49f8-b716-edd65266f926 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:41.660265 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b40b3d-6f5c-49f8-b716-edd65266f926 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:41.660265 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b40b3d-6f5c-49f8-b716-edd65266f926 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:41.662462 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 540/1099] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:41.671952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb86d2-efd4-4b9f-9a26-1da7a1e8b008 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:41.672770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb86d2-efd4-4b9f-9a26-1da7a1e8b008 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:41.673010 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb86d2-efd4-4b9f-9a26-1da7a1e8b008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:41.696882 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ffcb86d2-efd4-4b9f-9a26-1da7a1e8b008 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:41.698084 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ffcb86d2-efd4-4b9f-9a26-1da7a1e8b008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:41.770371 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 560/1100] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:41.804749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b40b3d-6f5c-49f8-b716-edd65266f926 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:41.897147 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:41.919243 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b40b3d-6f5c-49f8-b716-edd65266f926 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:41.953909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e4232-4232-489f-9cf3-d71f1034246d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:41.954659 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e4232-4232-489f-9cf3-d71f1034246d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:41.954937 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e4232-4232-489f-9cf3-d71f1034246d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:41.955211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e4232-4232-489f-9cf3-d71f1034246d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:41.966053 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e4232-4232-489f-9cf3-d71f1034246d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:41.973746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-3a5e4232-4232-489f-9cf3-d71f1034246d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:41.973746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:41.973746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:41.973746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:41.973746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:41.973746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:41.973746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:41.973746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:41.973746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:41.973746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:41.973746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:41.973746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:41.973746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:41.973746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:41.973746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:41.973746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:41.974827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:41.974827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:41.974827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:41.974827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:41.974827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:41.974827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:41.974827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:41.974827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:41.974827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:41.974827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:41.974827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:41.974827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:41.974827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:41.974827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:41.974827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:41.974827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:41.975824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:41.975824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:41.975824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:41.975824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:41.975824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:41.975824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:41.975824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:41.975824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:41.975824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:41.975824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:41.975824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:41.975824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:04:41.975824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 541/1101] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:04:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:41.981663 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb86d2-efd4-4b9f-9a26-1da7a1e8b008 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:41.985841 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:42.081775 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb86d2-efd4-4b9f-9a26-1da7a1e8b008 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:42.118520 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dac49-9263-4962-b4b2-3e8218f51eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:42.119184 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dac49-9263-4962-b4b2-3e8218f51eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:42.119459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dac49-9263-4962-b4b2-3e8218f51eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:42.119754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dac49-9263-4962-b4b2-3e8218f51eb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:42.120501 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95dac49-9263-4962-b4b2-3e8218f51eb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:42.120810 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95dac49-9263-4962-b4b2-3e8218f51eb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:42.122187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95dac49-9263-4962-b4b2-3e8218f51eb9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:42.133814 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 561/1102] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:04:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:42.140441 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e4232-4232-489f-9cf3-d71f1034246d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:42.264568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e4232-4232-489f-9cf3-d71f1034246d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:42.298302 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcdc019-af04-4f67-8362-578db023fdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:42.299174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdc019-af04-4f67-8362-578db023fdd4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:42.299455 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdc019-af04-4f67-8362-578db023fdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:42.299756 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdc019-af04-4f67-8362-578db023fdd4 None admin] PATH_INFO: `/v3/users/307303ec9d4244c398b884e73ff364a3` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:42.301076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcdc019-af04-4f67-8362-578db023fdd4 None admin] RBAC: Authorizing `identity:get_user(user_id=307303ec9d4244c398b884e73ff364a3)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:42.302532 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcdc019-af04-4f67-8362-578db023fdd4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:42.304178 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 542/1103] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:42 2022] GET /identity/v3/users/307303ec9d4244c398b884e73ff364a3 => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:42.312989 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dac49-9263-4962-b4b2-3e8218f51eb9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:42.383910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:42.421475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dac49-9263-4962-b4b2-3e8218f51eb9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:42.456081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320c8fc-baf4-445a-9845-1b9cf1c10412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:42.456723 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320c8fc-baf4-445a-9845-1b9cf1c10412 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:42.457007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320c8fc-baf4-445a-9845-1b9cf1c10412 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:42.457279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320c8fc-baf4-445a-9845-1b9cf1c10412 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:42.525327 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320c8fc-baf4-445a-9845-1b9cf1c10412 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:42.526734 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320c8fc-baf4-445a-9845-1b9cf1c10412 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:42.591749 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:42.629798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 562/1104] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:42.635960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcdc019-af04-4f67-8362-578db023fdd4 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:42.652098 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:42.742256 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcdc019-af04-4f67-8362-578db023fdd4 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:42.777054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf739a03-94a2-4f57-b0a2-89d3f0c9b74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:42.777844 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf739a03-94a2-4f57-b0a2-89d3f0c9b74b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:42.778120 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf739a03-94a2-4f57-b0a2-89d3f0c9b74b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:42.778389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf739a03-94a2-4f57-b0a2-89d3f0c9b74b None admin] PATH_INFO: `/v3/projects/2ef50902b46a4e29856a2bedb211310b` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:42.779650 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf739a03-94a2-4f57-b0a2-89d3f0c9b74b None admin] RBAC: Authorizing `identity:get_project(project_id=2ef50902b46a4e29856a2bedb211310b)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:42.781112 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf739a03-94a2-4f57-b0a2-89d3f0c9b74b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:42.783159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 543/1105] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:04:42 2022] GET /identity/v3/projects/2ef50902b46a4e29856a2bedb211310b => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:42.789743 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320c8fc-baf4-445a-9845-1b9cf1c10412 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:42.894404 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320c8fc-baf4-445a-9845-1b9cf1c10412 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:42.930244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae8de4-c60e-4e8c-88be-398f2062433a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:42.930894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae8de4-c60e-4e8c-88be-398f2062433a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:42.931176 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae8de4-c60e-4e8c-88be-398f2062433a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:42.931447 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae8de4-c60e-4e8c-88be-398f2062433a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:42.932309 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae8de4-c60e-4e8c-88be-398f2062433a None admin] RBAC: Adding query filter params (role.id=5c3c095bb53943eca95c4e049cd8cbf5, scope.project.id=2ef50902b46a4e29856a2bedb211310b, user.id=307303ec9d4244c398b884e73ff364a3) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:42.932619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae8de4-c60e-4e8c-88be-398f2062433a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:42.934060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae8de4-c60e-4e8c-88be-398f2062433a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:42.940120 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 563/1106] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:04:42 2022] GET /identity/v3/role_assignments?user.id=307303ec9d4244c398b884e73ff364a3&scope.project.id=2ef50902b46a4e29856a2bedb211310b&role.id=5c3c095bb53943eca95c4e049cd8cbf5 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:44.958586 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3847424-bce1-4ec0-9d53-9bfa619d972e None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:44.959142 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3847424-bce1-4ec0-9d53-9bfa619d972e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:44.959142 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3847424-bce1-4ec0-9d53-9bfa619d972e None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:44.961804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 544/1107] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:44.969831 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfd748-9c62-45dc-885d-633d497f60bf None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:44.970277 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfd748-9c62-45dc-885d-633d497f60bf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:44.970365 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfd748-9c62-45dc-885d-633d497f60bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:44.988730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-20bfd748-9c62-45dc-885d-633d497f60bf None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:44.989913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-20bfd748-9c62-45dc-885d-633d497f60bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:45.014005 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:45.059237 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 564/1108] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:45.083955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3847424-bce1-4ec0-9d53-9bfa619d972e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:45.176570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:45.199326 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3847424-bce1-4ec0-9d53-9bfa619d972e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:45.234947 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7a9b5-5191-4910-8ff7-913222ac27c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:45.235689 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7a9b5-5191-4910-8ff7-913222ac27c0 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:45.235973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7a9b5-5191-4910-8ff7-913222ac27c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:45.236244 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7a9b5-5191-4910-8ff7-913222ac27c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:45.237435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7a9b5-5191-4910-8ff7-913222ac27c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:45.238895 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7a9b5-5191-4910-8ff7-913222ac27c0 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:45.240530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 545/1109] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:45.247280 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfd748-9c62-45dc-885d-633d497f60bf None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:45.345701 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfd748-9c62-45dc-885d-633d497f60bf None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:45.380954 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028c5e4-6a3d-4c78-864a-436b23d318ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:45.381617 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028c5e4-6a3d-4c78-864a-436b23d318ac None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:45.381894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028c5e4-6a3d-4c78-864a-436b23d318ac None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:45.382165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028c5e4-6a3d-4c78-864a-436b23d318ac None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:45.382862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028c5e4-6a3d-4c78-864a-436b23d318ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:45.384342 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028c5e4-6a3d-4c78-864a-436b23d318ac None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:45.408341 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 565/1110] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Jun 27 21:04:45 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:47.491783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dbf94-d9ca-4357-86b3-888421d8cb7f None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:47.492411 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dbf94-d9ca-4357-86b3-888421d8cb7f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:47.492411 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dbf94-d9ca-4357-86b3-888421d8cb7f None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:47.493830 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 546/1111] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:47.500328 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a06791-2d41-4918-bf3d-ba00ef6bb4f3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:47.501049 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a06791-2d41-4918-bf3d-ba00ef6bb4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:47.501200 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a06791-2d41-4918-bf3d-ba00ef6bb4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:47.528648 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-69a06791-2d41-4918-bf3d-ba00ef6bb4f3 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:47.530679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-69a06791-2d41-4918-bf3d-ba00ef6bb4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:47.542016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:47.601675 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 566/1112] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:47.629242 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dbf94-d9ca-4357-86b3-888421d8cb7f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:47.738538 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dbf94-d9ca-4357-86b3-888421d8cb7f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:47.774291 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b40957-5ab7-44dc-a062-255fdbc9bdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:47.775141 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b40957-5ab7-44dc-a062-255fdbc9bdb7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:47.775423 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b40957-5ab7-44dc-a062-255fdbc9bdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:47.775726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b40957-5ab7-44dc-a062-255fdbc9bdb7 None admin] PATH_INFO: `/v3/roles/b3efb2b2e43c4af8b1cf36a37f1e05f9` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:47.777267 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b40957-5ab7-44dc-a062-255fdbc9bdb7 None admin] RBAC: Authorizing `identity:get_role(role_id=b3efb2b2e43c4af8b1cf36a37f1e05f9)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:47.778668 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b40957-5ab7-44dc-a062-255fdbc9bdb7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:47.780071 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 547/1113] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:47 2022] GET /identity/v3/roles/b3efb2b2e43c4af8b1cf36a37f1e05f9 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:47.789511 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a06791-2d41-4918-bf3d-ba00ef6bb4f3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:47.897108 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a06791-2d41-4918-bf3d-ba00ef6bb4f3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:47.934372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86cd726-65d1-465e-afd0-d0c17a8f9ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:47.935251 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cd726-65d1-465e-afd0-d0c17a8f9ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:47.935526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cd726-65d1-465e-afd0-d0c17a8f9ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:47.935832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cd726-65d1-465e-afd0-d0c17a8f9ce4 None admin] PATH_INFO: `/v3/users/a20d9c40b97e452489e6a41637bef6de` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:47.949432 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86cd726-65d1-465e-afd0-d0c17a8f9ce4 None admin] RBAC: Authorizing `identity:get_user(user_id=a20d9c40b97e452489e6a41637bef6de)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:47.950881 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86cd726-65d1-465e-afd0-d0c17a8f9ce4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:47.952637 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 567/1114] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:47 2022] GET /identity/v3/users/a20d9c40b97e452489e6a41637bef6de => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:47.959161 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b40957-5ab7-44dc-a062-255fdbc9bdb7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:47.985261 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:48.069352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b40957-5ab7-44dc-a062-255fdbc9bdb7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:48.104380 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ace04a-ff20-4bab-9e32-0f500fddea24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:48.105024 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ace04a-ff20-4bab-9e32-0f500fddea24 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:48.105299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ace04a-ff20-4bab-9e32-0f500fddea24 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:48.105571 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ace04a-ff20-4bab-9e32-0f500fddea24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:48.179087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ace04a-ff20-4bab-9e32-0f500fddea24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:48.180580 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ace04a-ff20-4bab-9e32-0f500fddea24 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:48.197862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:48.289710 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 548/1115] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:48.299334 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86cd726-65d1-465e-afd0-d0c17a8f9ce4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:48.305620 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:48.418026 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86cd726-65d1-465e-afd0-d0c17a8f9ce4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:48.453673 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc4e15-ab1d-4fd3-a37b-a61c7113a13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:48.454575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc4e15-ab1d-4fd3-a37b-a61c7113a13b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:48.454854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc4e15-ab1d-4fd3-a37b-a61c7113a13b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:48.455124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc4e15-ab1d-4fd3-a37b-a61c7113a13b None admin] PATH_INFO: `/v3/projects/2ef50902b46a4e29856a2bedb211310b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:48.456478 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc4e15-ab1d-4fd3-a37b-a61c7113a13b None admin] RBAC: Authorizing `identity:get_project(project_id=2ef50902b46a4e29856a2bedb211310b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:48.457930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc4e15-ab1d-4fd3-a37b-a61c7113a13b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:48.459638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 568/1116] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:04:48 2022] GET /identity/v3/projects/2ef50902b46a4e29856a2bedb211310b => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:48.469815 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ace04a-ff20-4bab-9e32-0f500fddea24 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:48.585828 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ace04a-ff20-4bab-9e32-0f500fddea24 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:48.603281 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:48.623622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821dc732-c318-410e-bdd3-6d4de823ad27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:48.624327 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821dc732-c318-410e-bdd3-6d4de823ad27 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:48.624607 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821dc732-c318-410e-bdd3-6d4de823ad27 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:48.624883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821dc732-c318-410e-bdd3-6d4de823ad27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:48.625712 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821dc732-c318-410e-bdd3-6d4de823ad27 None admin] RBAC: Adding query filter params (role.id=b3efb2b2e43c4af8b1cf36a37f1e05f9, scope.project.id=2ef50902b46a4e29856a2bedb211310b, user.id=a20d9c40b97e452489e6a41637bef6de) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:48.626026 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821dc732-c318-410e-bdd3-6d4de823ad27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:48.627478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821dc732-c318-410e-bdd3-6d4de823ad27 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:48.633098 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 549/1117] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:04:48 2022] GET /identity/v3/role_assignments?user.id=a20d9c40b97e452489e6a41637bef6de&scope.project.id=2ef50902b46a4e29856a2bedb211310b&role.id=b3efb2b2e43c4af8b1cf36a37f1e05f9 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:50.612949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ea523-8cb2-4842-8a34-4e1e7391ea6a None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:50.613803 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ea523-8cb2-4842-8a34-4e1e7391ea6a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:50.613803 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ea523-8cb2-4842-8a34-4e1e7391ea6a None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:50.616067 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 569/1118] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:50.621502 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838db6b-8054-4141-a6bd-9cbbe534b84f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:50.621945 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838db6b-8054-4141-a6bd-9cbbe534b84f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:50.622043 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838db6b-8054-4141-a6bd-9cbbe534b84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:50.641314 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e838db6b-8054-4141-a6bd-9cbbe534b84f None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:50.642529 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e838db6b-8054-4141-a6bd-9cbbe534b84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:50.716115 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 550/1119] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:50.739599 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ea523-8cb2-4842-8a34-4e1e7391ea6a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:50.820272 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:50.851418 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ea523-8cb2-4842-8a34-4e1e7391ea6a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:50.886534 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49eb1db-1da9-40e8-bc2e-d747d859f04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:50.887329 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49eb1db-1da9-40e8-bc2e-d747d859f04b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:50.887632 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49eb1db-1da9-40e8-bc2e-d747d859f04b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:50.887899 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49eb1db-1da9-40e8-bc2e-d747d859f04b None admin] PATH_INFO: `/v3/roles/b3efb2b2e43c4af8b1cf36a37f1e05f9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:50.889489 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49eb1db-1da9-40e8-bc2e-d747d859f04b None admin] RBAC: Authorizing `identity:get_role(role_id=b3efb2b2e43c4af8b1cf36a37f1e05f9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:50.891002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49eb1db-1da9-40e8-bc2e-d747d859f04b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:50.892417 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 570/1120] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:50 2022] GET /identity/v3/roles/b3efb2b2e43c4af8b1cf36a37f1e05f9 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:50.898664 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e838db6b-8054-4141-a6bd-9cbbe534b84f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:51.000620 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e838db6b-8054-4141-a6bd-9cbbe534b84f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:51.019221 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:51.046272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde19bc-6ead-4729-bf26-0050f3299efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:51.047201 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde19bc-6ead-4729-bf26-0050f3299efc None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:51.047466 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde19bc-6ead-4729-bf26-0050f3299efc None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:51.047775 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde19bc-6ead-4729-bf26-0050f3299efc None admin] PATH_INFO: `/v3/users/a20d9c40b97e452489e6a41637bef6de` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:51.049002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde19bc-6ead-4729-bf26-0050f3299efc None admin] RBAC: Authorizing `identity:get_user(user_id=a20d9c40b97e452489e6a41637bef6de)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:51.050455 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde19bc-6ead-4729-bf26-0050f3299efc None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:51.052186 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 551/1121] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:50 2022] GET /identity/v3/users/a20d9c40b97e452489e6a41637bef6de => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:51.058893 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49eb1db-1da9-40e8-bc2e-d747d859f04b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:51.182004 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49eb1db-1da9-40e8-bc2e-d747d859f04b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:51.216735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca12242-dbf8-40e5-9124-43e04a6b9e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:51.217595 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca12242-dbf8-40e5-9124-43e04a6b9e6a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:51.217938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca12242-dbf8-40e5-9124-43e04a6b9e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:51.218211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca12242-dbf8-40e5-9124-43e04a6b9e6a None admin] PATH_INFO: `/v3/projects/2ef50902b46a4e29856a2bedb211310b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:51.219436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca12242-dbf8-40e5-9124-43e04a6b9e6a None admin] RBAC: Authorizing `identity:get_project(project_id=2ef50902b46a4e29856a2bedb211310b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:51.220923 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca12242-dbf8-40e5-9124-43e04a6b9e6a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:51.222629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 571/1122] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:04:51 2022] GET /identity/v3/projects/2ef50902b46a4e29856a2bedb211310b => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:51.229203 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde19bc-6ead-4729-bf26-0050f3299efc None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:51.338318 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde19bc-6ead-4729-bf26-0050f3299efc None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:51.372517 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911cd70-d8a8-429a-8a24-bd241ab0af18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:51.373462 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911cd70-d8a8-429a-8a24-bd241ab0af18 None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:51.373739 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911cd70-d8a8-429a-8a24-bd241ab0af18 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:51.374050 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911cd70-d8a8-429a-8a24-bd241ab0af18 None admin] PATH_INFO: `/v3/projects/2ef50902b46a4e29856a2bedb211310b/users/a20d9c40b97e452489e6a41637bef6de/roles/b3efb2b2e43c4af8b1cf36a37f1e05f9` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:51.376420 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3911cd70-d8a8-429a-8a24-bd241ab0af18 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ef50902b46a4e29856a2bedb211310b, user_id=a20d9c40b97e452489e6a41637bef6de, role_id=b3efb2b2e43c4af8b1cf36a37f1e05f9)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:51.377856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3911cd70-d8a8-429a-8a24-bd241ab0af18 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:51.393481 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 552/1123] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:04:51 2022] PUT /identity/v3/projects/2ef50902b46a4e29856a2bedb211310b/users/a20d9c40b97e452489e6a41637bef6de/roles/b3efb2b2e43c4af8b1cf36a37f1e05f9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:04:53.383014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c13433-49fb-4248-865b-f57dd3e5c945 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:53.383632 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c13433-49fb-4248-865b-f57dd3e5c945 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:53.383632 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c13433-49fb-4248-865b-f57dd3e5c945 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:53.384981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 572/1124] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:53.391462 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502277e6-6ce1-49b7-8c1c-2788fd65c41b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:53.392221 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502277e6-6ce1-49b7-8c1c-2788fd65c41b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:53.392312 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502277e6-6ce1-49b7-8c1c-2788fd65c41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:53.415501 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-502277e6-6ce1-49b7-8c1c-2788fd65c41b None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:53.416743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-502277e6-6ce1-49b7-8c1c-2788fd65c41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:53.449440 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:53.491146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 553/1125] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:53.516163 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c13433-49fb-4248-865b-f57dd3e5c945 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:53.532334 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:53.635950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c13433-49fb-4248-865b-f57dd3e5c945 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:53.675772 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed5c9a-1615-488e-8910-a827cee677c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:53.676653 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed5c9a-1615-488e-8910-a827cee677c1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:53.676935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed5c9a-1615-488e-8910-a827cee677c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:53.677209 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed5c9a-1615-488e-8910-a827cee677c1 None admin] PATH_INFO: `/v3/roles/b3efb2b2e43c4af8b1cf36a37f1e05f9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:53.678832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed5c9a-1615-488e-8910-a827cee677c1 None admin] RBAC: Authorizing `identity:get_role(role_id=b3efb2b2e43c4af8b1cf36a37f1e05f9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:53.680267 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed5c9a-1615-488e-8910-a827cee677c1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:53.681675 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 573/1126] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:53 2022] GET /identity/v3/roles/b3efb2b2e43c4af8b1cf36a37f1e05f9 => generated 223 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:53.689815 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502277e6-6ce1-49b7-8c1c-2788fd65c41b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:53.803590 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502277e6-6ce1-49b7-8c1c-2788fd65c41b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:53.840982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fef86-7af5-491b-b20b-00ecc4ca8a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:53.841858 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fef86-7af5-491b-b20b-00ecc4ca8a9c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:53.842133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fef86-7af5-491b-b20b-00ecc4ca8a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:53.842403 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fef86-7af5-491b-b20b-00ecc4ca8a9c None admin] PATH_INFO: `/v3/users/a20d9c40b97e452489e6a41637bef6de` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:53.843769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fef86-7af5-491b-b20b-00ecc4ca8a9c None admin] RBAC: Authorizing `identity:get_user(user_id=a20d9c40b97e452489e6a41637bef6de)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:53.845226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fef86-7af5-491b-b20b-00ecc4ca8a9c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:53.846872 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 554/1127] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:53 2022] GET /identity/v3/users/a20d9c40b97e452489e6a41637bef6de => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:53.853192 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed5c9a-1615-488e-8910-a827cee677c1 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:53.939767 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:53.967072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed5c9a-1615-488e-8910-a827cee677c1 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:54.006005 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9b3b9-8aab-4d37-9167-f94e8dd9b836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:54.006672 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9b3b9-8aab-4d37-9167-f94e8dd9b836 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:54.006950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9b3b9-8aab-4d37-9167-f94e8dd9b836 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:54.007214 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9b3b9-8aab-4d37-9167-f94e8dd9b836 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:54.082980 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9b3b9-8aab-4d37-9167-f94e8dd9b836 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:54.084498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9b3b9-8aab-4d37-9167-f94e8dd9b836 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:54.173567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:54.204012 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 574/1128] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:54.210215 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fef86-7af5-491b-b20b-00ecc4ca8a9c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:54.246744 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:54.317615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fef86-7af5-491b-b20b-00ecc4ca8a9c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:54.354663 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a02cf-fdc3-40d4-a7ec-04b91e59bfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:54.355528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a02cf-fdc3-40d4-a7ec-04b91e59bfe6 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:54.355842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a02cf-fdc3-40d4-a7ec-04b91e59bfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:54.356119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a02cf-fdc3-40d4-a7ec-04b91e59bfe6 None admin] PATH_INFO: `/v3/projects/2ef50902b46a4e29856a2bedb211310b` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:54.357419 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a02cf-fdc3-40d4-a7ec-04b91e59bfe6 None admin] RBAC: Authorizing `identity:get_project(project_id=2ef50902b46a4e29856a2bedb211310b)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:54.358919 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a02cf-fdc3-40d4-a7ec-04b91e59bfe6 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:54.360676 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 555/1129] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:04:54 2022] GET /identity/v3/projects/2ef50902b46a4e29856a2bedb211310b => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:54.367212 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9b3b9-8aab-4d37-9167-f94e8dd9b836 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:54.476400 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9b3b9-8aab-4d37-9167-f94e8dd9b836 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:54.512030 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc10f08-cadb-4c45-a9a7-f2eb190c1600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:54.512677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc10f08-cadb-4c45-a9a7-f2eb190c1600 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:54.512964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc10f08-cadb-4c45-a9a7-f2eb190c1600 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:54.513239 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc10f08-cadb-4c45-a9a7-f2eb190c1600 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:54.514072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc10f08-cadb-4c45-a9a7-f2eb190c1600 None admin] RBAC: Adding query filter params (role.id=b3efb2b2e43c4af8b1cf36a37f1e05f9, scope.project.id=2ef50902b46a4e29856a2bedb211310b, user.id=a20d9c40b97e452489e6a41637bef6de) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:54.514388 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc10f08-cadb-4c45-a9a7-f2eb190c1600 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:54.515911 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc10f08-cadb-4c45-a9a7-f2eb190c1600 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:54.521947 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 575/1130] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:04:54 2022] GET /identity/v3/role_assignments?user.id=a20d9c40b97e452489e6a41637bef6de&scope.project.id=2ef50902b46a4e29856a2bedb211310b&role.id=b3efb2b2e43c4af8b1cf36a37f1e05f9 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:56.537831 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc2ae1-c003-4e8d-9748-6065d289275a None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:56.538415 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc2ae1-c003-4e8d-9748-6065d289275a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:56.538415 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc2ae1-c003-4e8d-9748-6065d289275a None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:56.539784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 556/1131] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:56.546083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443d4a9-6685-46ee-8c65-96ab24b0421f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:56.546513 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443d4a9-6685-46ee-8c65-96ab24b0421f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:56.546648 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443d4a9-6685-46ee-8c65-96ab24b0421f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:56.570297 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b443d4a9-6685-46ee-8c65-96ab24b0421f None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:56.571575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b443d4a9-6685-46ee-8c65-96ab24b0421f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:56.611563 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:56.662480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 576/1132] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:56.695386 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc2ae1-c003-4e8d-9748-6065d289275a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:56.807224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc2ae1-c003-4e8d-9748-6065d289275a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:56.809827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:56.842750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1443811-dbc1-4736-a05c-ed45a2c85b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:56.843534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1443811-dbc1-4736-a05c-ed45a2c85b94 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:56.843846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1443811-dbc1-4736-a05c-ed45a2c85b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:56.844122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1443811-dbc1-4736-a05c-ed45a2c85b94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:56.845340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1443811-dbc1-4736-a05c-ed45a2c85b94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:56.846773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1443811-dbc1-4736-a05c-ed45a2c85b94 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:56.848401 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 557/1133] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:56.854383 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443d4a9-6685-46ee-8c65-96ab24b0421f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:56.961978 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443d4a9-6685-46ee-8c65-96ab24b0421f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:57.000099 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64e7b0-2fb0-4c63-a1d5-766e850d22b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:57.001081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64e7b0-2fb0-4c63-a1d5-766e850d22b8 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:57.001456 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64e7b0-2fb0-4c63-a1d5-766e850d22b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:57.001808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64e7b0-2fb0-4c63-a1d5-766e850d22b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:57.002877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64e7b0-2fb0-4c63-a1d5-766e850d22b8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:57.005162 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64e7b0-2fb0-4c63-a1d5-766e850d22b8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:57.036323 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 577/1134] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Jun 27 21:04:56 2022] POST /identity/v3/projects => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:59.052390 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b7484-6bed-45ce-bcbd-1276e5d52e69 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:59.052915 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b7484-6bed-45ce-bcbd-1276e5d52e69 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:59.052915 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b7484-6bed-45ce-bcbd-1276e5d52e69 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:59.054317 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 558/1135] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:59.060715 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d23abc-04a6-4d6c-8f10-8d9fb91d9d4f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:59.061499 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d23abc-04a6-4d6c-8f10-8d9fb91d9d4f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:59.061703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d23abc-04a6-4d6c-8f10-8d9fb91d9d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:59.085948 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-34d23abc-04a6-4d6c-8f10-8d9fb91d9d4f None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:59.087158 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-34d23abc-04a6-4d6c-8f10-8d9fb91d9d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:59.107136 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:59.172835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 578/1136] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:59.197663 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b7484-6bed-45ce-bcbd-1276e5d52e69 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:59.311937 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b7484-6bed-45ce-bcbd-1276e5d52e69 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:59.347188 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471324a-3192-4d4e-8fb1-be3070ad525b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:59.348010 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471324a-3192-4d4e-8fb1-be3070ad525b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:59.348289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471324a-3192-4d4e-8fb1-be3070ad525b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:59.348560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471324a-3192-4d4e-8fb1-be3070ad525b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:59.349843 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471324a-3192-4d4e-8fb1-be3070ad525b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:59.351294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471324a-3192-4d4e-8fb1-be3070ad525b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:59.352945 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 559/1137] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:59.359107 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d23abc-04a6-4d6c-8f10-8d9fb91d9d4f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:59.469265 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d23abc-04a6-4d6c-8f10-8d9fb91d9d4f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:59.504096 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b9cc4-213e-4b88-af04-b1bf1970b476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:59.504756 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b9cc4-213e-4b88-af04-b1bf1970b476 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:59.505033 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b9cc4-213e-4b88-af04-b1bf1970b476 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:59.505305 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b9cc4-213e-4b88-af04-b1bf1970b476 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:59.506006 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b9cc4-213e-4b88-af04-b1bf1970b476 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:59.507460 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b9cc4-213e-4b88-af04-b1bf1970b476 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:59.533311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 579/1138] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Jun 27 21:04:59 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:05:01.575248 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb3f37-623c-48cc-ac5f-0cbf5af8e4a6 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:01.575866 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb3f37-623c-48cc-ac5f-0cbf5af8e4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:01.575866 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb3f37-623c-48cc-ac5f-0cbf5af8e4a6 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:01.577261 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 560/1139] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:05:01.583571 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52838cfb-7810-4f55-8ced-54dd5548c2e9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:01.584270 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52838cfb-7810-4f55-8ced-54dd5548c2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:01.584515 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52838cfb-7810-4f55-8ced-54dd5548c2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:01.611079 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-52838cfb-7810-4f55-8ced-54dd5548c2e9 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:05:01.612303 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-52838cfb-7810-4f55-8ced-54dd5548c2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:05:01.614219 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:01.686086 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 580/1140] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:05:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:05:01.710033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb3f37-623c-48cc-ac5f-0cbf5af8e4a6 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:01.758413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:01.818485 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb3f37-623c-48cc-ac5f-0cbf5af8e4a6 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:01.853477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b8a23-a16b-4758-801c-1ff24a8eb651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:01.854199 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b8a23-a16b-4758-801c-1ff24a8eb651 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:01.854480 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b8a23-a16b-4758-801c-1ff24a8eb651 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:01.854747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b8a23-a16b-4758-801c-1ff24a8eb651 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:01.865528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b8a23-a16b-4758-801c-1ff24a8eb651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:05:01.873184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-6e0b8a23-a16b-4758-801c-1ff24a8eb651 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:05:01.873184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:05:01.873184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:05:01.873184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:05:01.873184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:05:01.873184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:05:01.873184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:05:01.873184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:05:01.873184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:05:01.873184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:05:01.873184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:05:01.873184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:05:01.873184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:05:01.873184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:05:01.873184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:05:01.873184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:05:01.874360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:05:01.874360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:05:01.874360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:05:01.874360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:05:01.874360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:05:01.874360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:05:01.874360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:05:01.874360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:05:01.874360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:05:01.874360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:05:01.874360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:05:01.874360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:05:01.874360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:05:01.874360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:05:01.874360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:05:01.874360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:05:01.875587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:05:01.875587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:05:01.875587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:05:01.875587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:05:01.875587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:05:01.875587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:05:01.875587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:05:01.875587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:05:01.875587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:05:01.875587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:05:01.875587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:05:01.875587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:05:01.875587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 561/1141] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:05:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:05:01.882253 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52838cfb-7810-4f55-8ced-54dd5548c2e9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:01.992028 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52838cfb-7810-4f55-8ced-54dd5548c2e9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:02.034659 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee88e62-a0be-4d12-9561-4166aad9704b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:02.035336 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee88e62-a0be-4d12-9561-4166aad9704b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:02.035651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee88e62-a0be-4d12-9561-4166aad9704b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:02.035917 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee88e62-a0be-4d12-9561-4166aad9704b None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:02.036670 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee88e62-a0be-4d12-9561-4166aad9704b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:05:02.036979 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee88e62-a0be-4d12-9561-4166aad9704b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:02.038373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee88e62-a0be-4d12-9561-4166aad9704b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:02.044010 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:02.048163 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 581/1142] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:05:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:02.054996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b8a23-a16b-4758-801c-1ff24a8eb651 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:02.176760 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:02.179883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b8a23-a16b-4758-801c-1ff24a8eb651 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:02.213691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74679d67-af24-4685-ad10-6738dce21a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:02.214573 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74679d67-af24-4685-ad10-6738dce21a6e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:02.214850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74679d67-af24-4685-ad10-6738dce21a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:02.215125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74679d67-af24-4685-ad10-6738dce21a6e None admin] PATH_INFO: `/v3/users/8890bf01e91d4462aed0c9c9cddbdf32` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:02.228773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74679d67-af24-4685-ad10-6738dce21a6e None admin] RBAC: Authorizing `identity:get_user(user_id=8890bf01e91d4462aed0c9c9cddbdf32)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:02.230224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74679d67-af24-4685-ad10-6738dce21a6e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:02.231890 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 562/1143] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:05:02 2022] GET /identity/v3/users/8890bf01e91d4462aed0c9c9cddbdf32 => generated 286 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:02.237832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee88e62-a0be-4d12-9561-4166aad9704b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:02.341459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee88e62-a0be-4d12-9561-4166aad9704b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:02.378896 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623b944-06e0-4b98-a8c8-d558f321e2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:02.379545 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623b944-06e0-4b98-a8c8-d558f321e2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:02.379862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623b944-06e0-4b98-a8c8-d558f321e2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:02.380141 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623b944-06e0-4b98-a8c8-d558f321e2f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:02.457268 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:02.460126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623b944-06e0-4b98-a8c8-d558f321e2f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:02.461948 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623b944-06e0-4b98-a8c8-d558f321e2f2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:02.564986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 582/1144] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:05:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:05:02.571330 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74679d67-af24-4685-ad10-6738dce21a6e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:02.674039 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74679d67-af24-4685-ad10-6738dce21a6e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:02.724217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d204b-1959-446d-8487-e514c415f5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:02.725012 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d204b-1959-446d-8487-e514c415f5ff None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:02.725296 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d204b-1959-446d-8487-e514c415f5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:02.725572 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d204b-1959-446d-8487-e514c415f5ff None admin] PATH_INFO: `/v3/projects/5fb20df7172e44e8b0e48e0eefdbd85e` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:02.727286 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d204b-1959-446d-8487-e514c415f5ff None admin] RBAC: Authorizing `identity:get_project(project_id=5fb20df7172e44e8b0e48e0eefdbd85e)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:02.729724 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d204b-1959-446d-8487-e514c415f5ff None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:02.731431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 563/1145] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:05:02 2022] GET /identity/v3/projects/5fb20df7172e44e8b0e48e0eefdbd85e => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:02.738278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623b944-06e0-4b98-a8c8-d558f321e2f2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:02.835636 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:02.839025 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623b944-06e0-4b98-a8c8-d558f321e2f2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:02.876312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549161f5-9177-478c-a6cc-5f23ff24ea72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:02.876958 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549161f5-9177-478c-a6cc-5f23ff24ea72 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:02.877236 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549161f5-9177-478c-a6cc-5f23ff24ea72 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:02.877507 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549161f5-9177-478c-a6cc-5f23ff24ea72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:02.878331 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549161f5-9177-478c-a6cc-5f23ff24ea72 None admin] RBAC: Adding query filter params (role.id=5c3c095bb53943eca95c4e049cd8cbf5, scope.project.id=5fb20df7172e44e8b0e48e0eefdbd85e, user.id=8890bf01e91d4462aed0c9c9cddbdf32) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:05:02.878640 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549161f5-9177-478c-a6cc-5f23ff24ea72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:02.880119 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549161f5-9177-478c-a6cc-5f23ff24ea72 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:02.885685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 583/1146] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:05:02 2022] GET /identity/v3/role_assignments?user.id=8890bf01e91d4462aed0c9c9cddbdf32&scope.project.id=5fb20df7172e44e8b0e48e0eefdbd85e&role.id=5c3c095bb53943eca95c4e049cd8cbf5 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:04.850739 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60080f-e708-45b3-a847-ea13f20499ae None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:04.851302 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60080f-e708-45b3-a847-ea13f20499ae None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:04.851302 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60080f-e708-45b3-a847-ea13f20499ae None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:04.852684 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 564/1147] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:05:04.859089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08938b95-af03-4ff7-9f1c-9f2e979f976e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:04.859924 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08938b95-af03-4ff7-9f1c-9f2e979f976e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:04.860022 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08938b95-af03-4ff7-9f1c-9f2e979f976e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:04.879317 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-08938b95-af03-4ff7-9f1c-9f2e979f976e None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:05:04.880526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-08938b95-af03-4ff7-9f1c-9f2e979f976e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:05:04.948611 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 584/1148] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:05:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:05:04.978728 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60080f-e708-45b3-a847-ea13f20499ae None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:05.008036 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:05.092489 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60080f-e708-45b3-a847-ea13f20499ae None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:05.128256 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd066873-c5ac-44fb-af02-26ae4eebdd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:05.128998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd066873-c5ac-44fb-af02-26ae4eebdd20 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:05.129284 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd066873-c5ac-44fb-af02-26ae4eebdd20 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:05.129559 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd066873-c5ac-44fb-af02-26ae4eebdd20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:05.140744 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd066873-c5ac-44fb-af02-26ae4eebdd20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:05:05.148560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-fd066873-c5ac-44fb-af02-26ae4eebdd20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:05:05.148560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:05:05.148560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:05:05.148560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:05:05.148560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:05:05.148560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:05:05.148560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:05:05.148560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:05:05.148560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:05:05.148560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:05:05.148560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:05:05.148560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:05:05.148560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:05:05.148560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:05:05.148560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:05:05.148560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:05:05.149639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:05:05.149639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:05:05.149639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:05:05.149639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:05:05.149639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:05:05.149639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:05:05.149639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:05:05.149639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:05:05.149639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:05:05.149639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:05:05.149639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:05:05.149639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:05:05.149639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:05:05.149639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:05:05.149639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:05:05.149639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:05:05.150826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:05:05.150826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:05:05.150826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:05:05.150826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:05:05.150826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:05:05.150826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:05:05.150826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:05:05.150826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:05:05.150826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:05:05.150826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:05:05.150826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:05:05.150826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:05:05.150826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 565/1149] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:05:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:05:05.156279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08938b95-af03-4ff7-9f1c-9f2e979f976e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:05.252374 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:05.262257 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08938b95-af03-4ff7-9f1c-9f2e979f976e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:05.298080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958e011-73bb-4cdd-8cc9-117719597774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:05.298737 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958e011-73bb-4cdd-8cc9-117719597774 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:05.299017 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958e011-73bb-4cdd-8cc9-117719597774 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:05.299291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958e011-73bb-4cdd-8cc9-117719597774 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:05.300076 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958e011-73bb-4cdd-8cc9-117719597774 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:05:05.300386 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958e011-73bb-4cdd-8cc9-117719597774 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:05.301774 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958e011-73bb-4cdd-8cc9-117719597774 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:05.309909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 585/1150] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:05:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:05.320033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd066873-c5ac-44fb-af02-26ae4eebdd20 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:05.412191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:05.428368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd066873-c5ac-44fb-af02-26ae4eebdd20 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:05.463034 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865b4a3-c2f4-4e43-80d1-957ca5651c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:05.463924 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865b4a3-c2f4-4e43-80d1-957ca5651c7e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:05.464202 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865b4a3-c2f4-4e43-80d1-957ca5651c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:05.464464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865b4a3-c2f4-4e43-80d1-957ca5651c7e None admin] PATH_INFO: `/v3/users/8890bf01e91d4462aed0c9c9cddbdf32` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:05.465770 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865b4a3-c2f4-4e43-80d1-957ca5651c7e None admin] RBAC: Authorizing `identity:get_user(user_id=8890bf01e91d4462aed0c9c9cddbdf32)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:05.467236 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865b4a3-c2f4-4e43-80d1-957ca5651c7e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:05.468907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 566/1151] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:05:05 2022] GET /identity/v3/users/8890bf01e91d4462aed0c9c9cddbdf32 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:05.474892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958e011-73bb-4cdd-8cc9-117719597774 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:05.579953 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958e011-73bb-4cdd-8cc9-117719597774 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:05.617905 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e3f29-a706-4b66-a126-0c213687f127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:05.618774 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e3f29-a706-4b66-a126-0c213687f127 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:05.619055 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e3f29-a706-4b66-a126-0c213687f127 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:05.619331 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e3f29-a706-4b66-a126-0c213687f127 None admin] PATH_INFO: `/v3/projects/5fb20df7172e44e8b0e48e0eefdbd85e` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:05.620668 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e3f29-a706-4b66-a126-0c213687f127 None admin] RBAC: Authorizing `identity:get_project(project_id=5fb20df7172e44e8b0e48e0eefdbd85e)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:05.622120 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e3f29-a706-4b66-a126-0c213687f127 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:05.623982 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 586/1152] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:05:05 2022] GET /identity/v3/projects/5fb20df7172e44e8b0e48e0eefdbd85e => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:05.633385 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865b4a3-c2f4-4e43-80d1-957ca5651c7e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:05.753255 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865b4a3-c2f4-4e43-80d1-957ca5651c7e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:05.789104 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964a07a-8e7f-4a3b-bfa3-093429cfeb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:05.790002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964a07a-8e7f-4a3b-bfa3-093429cfeb2a None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:05.790282 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964a07a-8e7f-4a3b-bfa3-093429cfeb2a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:05.790559 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964a07a-8e7f-4a3b-bfa3-093429cfeb2a None admin] PATH_INFO: `/v3/projects/5fb20df7172e44e8b0e48e0eefdbd85e/users/8890bf01e91d4462aed0c9c9cddbdf32/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:05.792811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964a07a-8e7f-4a3b-bfa3-093429cfeb2a None admin] RBAC: Authorizing `identity:create_grant(project_id=5fb20df7172e44e8b0e48e0eefdbd85e, user_id=8890bf01e91d4462aed0c9c9cddbdf32, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:05.794258 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964a07a-8e7f-4a3b-bfa3-093429cfeb2a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:05.810433 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 567/1153] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:05:05 2022] PUT /identity/v3/projects/5fb20df7172e44e8b0e48e0eefdbd85e/users/8890bf01e91d4462aed0c9c9cddbdf32/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:05:07.768594 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109bab0-9a1b-4c0c-8227-e4cdcb4b57cd None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:07.769488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109bab0-9a1b-4c0c-8227-e4cdcb4b57cd None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:07.769488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109bab0-9a1b-4c0c-8227-e4cdcb4b57cd None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:07.771668 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 587/1154] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:05:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:05:07.780251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588c24c-ac13-40c8-a29d-64dfe32b4f0e None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:07.780642 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588c24c-ac13-40c8-a29d-64dfe32b4f0e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:07.780794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588c24c-ac13-40c8-a29d-64dfe32b4f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:07.800405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6588c24c-ac13-40c8-a29d-64dfe32b4f0e None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:05:07.801596 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6588c24c-ac13-40c8-a29d-64dfe32b4f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:05:07.812049 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:07.877799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 568/1155] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:05:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:05:07.913788 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109bab0-9a1b-4c0c-8227-e4cdcb4b57cd None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:07.944058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:08.025549 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109bab0-9a1b-4c0c-8227-e4cdcb4b57cd None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:08.062754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db970b0a-7252-458e-95b4-7a85a96bd305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:08.063554 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db970b0a-7252-458e-95b4-7a85a96bd305 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:08.063869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db970b0a-7252-458e-95b4-7a85a96bd305 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:08.064144 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db970b0a-7252-458e-95b4-7a85a96bd305 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:08.075346 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db970b0a-7252-458e-95b4-7a85a96bd305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:05:08.083075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-db970b0a-7252-458e-95b4-7a85a96bd305 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:05:08.083075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:05:08.083075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:05:08.083075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:05:08.083075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:05:08.083075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:05:08.083075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:05:08.083075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:05:08.083075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:05:08.083075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:05:08.083075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:05:08.083075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:05:08.083075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:05:08.083075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:05:08.083075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:05:08.083075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:05:08.084216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:05:08.084216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:05:08.084216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:05:08.084216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:05:08.084216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:05:08.084216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:05:08.084216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:05:08.084216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:05:08.084216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:05:08.084216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:05:08.084216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:05:08.084216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:05:08.084216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:05:08.084216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:05:08.084216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:05:08.084216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:05:08.085384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:05:08.085384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:05:08.085384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:05:08.085384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:05:08.085384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:05:08.085384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:05:08.085384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:05:08.085384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:05:08.085384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:05:08.085384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:05:08.085384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:05:08.085384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:05:08.085384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 588/1156] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:05:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:05:08.090549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588c24c-ac13-40c8-a29d-64dfe32b4f0e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:08.195128 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588c24c-ac13-40c8-a29d-64dfe32b4f0e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:08.231221 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932cf04-c8fb-42ea-81b0-19c76e0d34a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:08.231916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932cf04-c8fb-42ea-81b0-19c76e0d34a0 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:08.232201 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932cf04-c8fb-42ea-81b0-19c76e0d34a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:08.232477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932cf04-c8fb-42ea-81b0-19c76e0d34a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:08.233289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932cf04-c8fb-42ea-81b0-19c76e0d34a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:05:08.233604 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932cf04-c8fb-42ea-81b0-19c76e0d34a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:08.234990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932cf04-c8fb-42ea-81b0-19c76e0d34a0 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:08.243370 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 569/1157] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:05:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:08.251870 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db970b0a-7252-458e-95b4-7a85a96bd305 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:08.339847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:08.356840 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db970b0a-7252-458e-95b4-7a85a96bd305 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:08.395392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c458791-02fc-444b-9a4a-828c0163ad9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:08.397470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c458791-02fc-444b-9a4a-828c0163ad9b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:08.397754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c458791-02fc-444b-9a4a-828c0163ad9b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:08.398010 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c458791-02fc-444b-9a4a-828c0163ad9b None admin] PATH_INFO: `/v3/users/8890bf01e91d4462aed0c9c9cddbdf32` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:08.399350 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c458791-02fc-444b-9a4a-828c0163ad9b None admin] RBAC: Authorizing `identity:get_user(user_id=8890bf01e91d4462aed0c9c9cddbdf32)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:08.400833 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c458791-02fc-444b-9a4a-828c0163ad9b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:08.402486 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 589/1158] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:05:08 2022] GET /identity/v3/users/8890bf01e91d4462aed0c9c9cddbdf32 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:08.413135 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932cf04-c8fb-42ea-81b0-19c76e0d34a0 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:08.429162 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:08.540045 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932cf04-c8fb-42ea-81b0-19c76e0d34a0 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:08.575298 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526225cd-6a50-4f98-8e15-f4c0ba69a25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:08.575964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526225cd-6a50-4f98-8e15-f4c0ba69a25a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:08.576243 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526225cd-6a50-4f98-8e15-f4c0ba69a25a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:08.576568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526225cd-6a50-4f98-8e15-f4c0ba69a25a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:08.645195 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526225cd-6a50-4f98-8e15-f4c0ba69a25a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:08.646565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526225cd-6a50-4f98-8e15-f4c0ba69a25a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:08.654225 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:08.747436 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 570/1159] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:05:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:05:08.754286 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c458791-02fc-444b-9a4a-828c0163ad9b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:08.863630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c458791-02fc-444b-9a4a-828c0163ad9b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:08.908154 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ad585-0fca-42c0-957d-274fc96ac49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:08.908941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ad585-0fca-42c0-957d-274fc96ac49c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:08.909216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ad585-0fca-42c0-957d-274fc96ac49c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:08.909488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ad585-0fca-42c0-957d-274fc96ac49c None admin] PATH_INFO: `/v3/projects/5fb20df7172e44e8b0e48e0eefdbd85e` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:08.910726 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ad585-0fca-42c0-957d-274fc96ac49c None admin] RBAC: Authorizing `identity:get_project(project_id=5fb20df7172e44e8b0e48e0eefdbd85e)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:08.912202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ad585-0fca-42c0-957d-274fc96ac49c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:08.913941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 590/1160] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:05:08 2022] GET /identity/v3/projects/5fb20df7172e44e8b0e48e0eefdbd85e => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:08.921130 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526225cd-6a50-4f98-8e15-f4c0ba69a25a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:09.036200 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526225cd-6a50-4f98-8e15-f4c0ba69a25a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:09.043459 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:09.072309 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce991d-84ee-4341-bdd8-473e48f97377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:09.072968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce991d-84ee-4341-bdd8-473e48f97377 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:09.073250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce991d-84ee-4341-bdd8-473e48f97377 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:09.073524 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce991d-84ee-4341-bdd8-473e48f97377 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:09.074354 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce991d-84ee-4341-bdd8-473e48f97377 None admin] RBAC: Adding query filter params (role.id=5c3c095bb53943eca95c4e049cd8cbf5, scope.project.id=5fb20df7172e44e8b0e48e0eefdbd85e, user.id=8890bf01e91d4462aed0c9c9cddbdf32) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:05:09.074665 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce991d-84ee-4341-bdd8-473e48f97377 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:09.076149 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce991d-84ee-4341-bdd8-473e48f97377 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:09.082217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 571/1161] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:05:08 2022] GET /identity/v3/role_assignments?user.id=8890bf01e91d4462aed0c9c9cddbdf32&scope.project.id=5fb20df7172e44e8b0e48e0eefdbd85e&role.id=5c3c095bb53943eca95c4e049cd8cbf5 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:11.104309 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f83c70-c761-4885-84d2-6d1c62281a68 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:11.104858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f83c70-c761-4885-84d2-6d1c62281a68 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:11.104858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f83c70-c761-4885-84d2-6d1c62281a68 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:11.106246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 591/1162] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:05:11.112633 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddaa39-fe40-44d5-b393-782126adb554 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:11.113321 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddaa39-fe40-44d5-b393-782126adb554 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:11.113500 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddaa39-fe40-44d5-b393-782126adb554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:11.141384 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b4ddaa39-fe40-44d5-b393-782126adb554 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:05:11.142595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b4ddaa39-fe40-44d5-b393-782126adb554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:05:11.217120 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 572/1163] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:05:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:05:11.245902 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f83c70-c761-4885-84d2-6d1c62281a68 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:11.270240 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:11.353073 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f83c70-c761-4885-84d2-6d1c62281a68 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:11.388276 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944685e-0fbc-40a5-a34f-85151a30cb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:11.388987 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944685e-0fbc-40a5-a34f-85151a30cb69 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:11.389265 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944685e-0fbc-40a5-a34f-85151a30cb69 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:11.389537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944685e-0fbc-40a5-a34f-85151a30cb69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:11.459312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944685e-0fbc-40a5-a34f-85151a30cb69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:11.460797 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944685e-0fbc-40a5-a34f-85151a30cb69 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:11.481355 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:11.561464 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 592/1164] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:05:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:05:11.567720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ddaa39-fe40-44d5-b393-782126adb554 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:11.668770 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:11.672111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ddaa39-fe40-44d5-b393-782126adb554 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:11.719767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f5f7a-56ec-4471-94a3-e438440b70ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:11.720570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f5f7a-56ec-4471-94a3-e438440b70ca None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:11.720847 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f5f7a-56ec-4471-94a3-e438440b70ca None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:11.721124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f5f7a-56ec-4471-94a3-e438440b70ca None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:11.728419 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f5f7a-56ec-4471-94a3-e438440b70ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:11.729884 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f5f7a-56ec-4471-94a3-e438440b70ca None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:11.744469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-332f5f7a-56ec-4471-94a3-e438440b70ca None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 27 21:05:11.744469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:05:11.744469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 27 21:05:11.744469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 27 21:05:11.744469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:05:11.744469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:05:11.744469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:05:11.744469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:05:11.744469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 27 21:05:11.744469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:05:11.744469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:05:11.744469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:05:11.744469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:05:11.744469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:05:11.744469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:05:11.744469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:05:11.744469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:05:11.745617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:05:11.745617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:05:11.745617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:05:11.745617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:05:11.745617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:05:11.745617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:05:11.745617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:05:11.745617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:05:11.745617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:05:11.745617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:05:11.745617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:05:11.745617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:05:11.745617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:05:11.745617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:05:11.745617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:05:11.745617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:05:11.746675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:05:11.746675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:05:11.746675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:05:11.746675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:05:11.746675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:05:11.746675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:05:11.746675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:05:11.746675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:05:11.746675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:05:11.746675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:05:11.746675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 27 21:05:11.746675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:05:11.746675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 27 21:05:11.746675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:05:11.746675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 573/1165] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Jun 27 21:05:11 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:05:11.751760 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944685e-0fbc-40a5-a34f-85151a30cb69 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:11.861797 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944685e-0fbc-40a5-a34f-85151a30cb69 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:11.883270 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:11.900706 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d51a0-1686-4a7f-9d78-d6296f5d525a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:11.901375 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d51a0-1686-4a7f-9d78-d6296f5d525a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:11.901665 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d51a0-1686-4a7f-9d78-d6296f5d525a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:11.901939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d51a0-1686-4a7f-9d78-d6296f5d525a None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:11.902667 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d51a0-1686-4a7f-9d78-d6296f5d525a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:05:11.902975 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d51a0-1686-4a7f-9d78-d6296f5d525a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:11.904396 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d51a0-1686-4a7f-9d78-d6296f5d525a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:11.912755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 593/1166] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Jun 27 21:05:11 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:13.874426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff51de-1614-493b-88f8-05aa652d249d None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:13.875281 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff51de-1614-493b-88f8-05aa652d249d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:13.875281 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff51de-1614-493b-88f8-05aa652d249d None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:13.877540 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 574/1167] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:05:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:05:13.886311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b98a5-38c2-44f3-a2e1-6562930e4130 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:13.886790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b98a5-38c2-44f3-a2e1-6562930e4130 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:13.886890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b98a5-38c2-44f3-a2e1-6562930e4130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:13.911930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-299b98a5-38c2-44f3-a2e1-6562930e4130 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:05:13.913133 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-299b98a5-38c2-44f3-a2e1-6562930e4130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:05:13.984720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 594/1168] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:05:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:05:14.011563 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff51de-1614-493b-88f8-05aa652d249d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:14.122619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff51de-1614-493b-88f8-05aa652d249d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:14.157921 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70acf14-c0dd-457e-b983-4df077c5d09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:14.158501 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70acf14-c0dd-457e-b983-4df077c5d09d None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:14.158778 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70acf14-c0dd-457e-b983-4df077c5d09d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:14.159047 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70acf14-c0dd-457e-b983-4df077c5d09d None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:14.159814 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70acf14-c0dd-457e-b983-4df077c5d09d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:14.161189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70acf14-c0dd-457e-b983-4df077c5d09d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:14.179962 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 575/1169] 10.222.0.27 () {60 vars in 1138 bytes} [Mon Jun 27 21:05:14 2022] POST /identity/v3/domains => generated 267 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:05:16.242247 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b7a3e-127e-454f-a840-52939b4145e6 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:16.242870 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b7a3e-127e-454f-a840-52939b4145e6 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:16.242870 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b7a3e-127e-454f-a840-52939b4145e6 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:16.244256 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 595/1170] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:05:16.250690 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cef95-849c-4029-bbb1-43f0c5004076 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:16.251466 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cef95-849c-4029-bbb1-43f0c5004076 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:16.251591 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cef95-849c-4029-bbb1-43f0c5004076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:16.274558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-fd9cef95-849c-4029-bbb1-43f0c5004076 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:05:16.275793 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fd9cef95-849c-4029-bbb1-43f0c5004076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:05:16.292467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:16.360854 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 576/1171] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:05:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:05:16.384907 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b7a3e-127e-454f-a840-52939b4145e6 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:16.516235 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b7a3e-127e-454f-a840-52939b4145e6 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:16.532657 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:16.556770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e22be2-e896-4ef3-a565-090484114eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:16.557634 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e22be2-e896-4ef3-a565-090484114eeb None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:16.557907 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e22be2-e896-4ef3-a565-090484114eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:16.558174 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e22be2-e896-4ef3-a565-090484114eeb None admin] PATH_INFO: `/v3/domains/91a46aa676bc445fa166f8561029562d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:16.568884 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e22be2-e896-4ef3-a565-090484114eeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=91a46aa676bc445fa166f8561029562d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:16.570359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e22be2-e896-4ef3-a565-090484114eeb None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:16.572020 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 596/1172] 10.222.0.27 () {56 vars in 1183 bytes} [Mon Jun 27 21:05:16 2022] GET /identity/v3/domains/91a46aa676bc445fa166f8561029562d => generated 267 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:16.578879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cef95-849c-4029-bbb1-43f0c5004076 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:16.694144 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cef95-849c-4029-bbb1-43f0c5004076 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:16.732001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf413d-2d4c-49e2-99e9-629f9d28e1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:16.732703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf413d-2d4c-49e2-99e9-629f9d28e1ed None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:16.732987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf413d-2d4c-49e2-99e9-629f9d28e1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:16.733270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf413d-2d4c-49e2-99e9-629f9d28e1ed None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:16.733967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf413d-2d4c-49e2-99e9-629f9d28e1ed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:16.735413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf413d-2d4c-49e2-99e9-629f9d28e1ed None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:16.768916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 577/1173] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Jun 27 21:05:16 2022] POST /identity/v3/projects => generated 361 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:05:18.817169 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8a261-070e-4356-ad6f-e05f270771ad None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:18.818254 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8a261-070e-4356-ad6f-e05f270771ad None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:18.818254 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8a261-070e-4356-ad6f-e05f270771ad None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:18.820345 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 597/1174] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:05:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:05:18.828280 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52d7cb-174e-4522-81d5-94bf3fbffb97 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:18.828688 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52d7cb-174e-4522-81d5-94bf3fbffb97 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:18.828841 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52d7cb-174e-4522-81d5-94bf3fbffb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:18.834141 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:18.848953 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-7b52d7cb-174e-4522-81d5-94bf3fbffb97 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:05:18.850160 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7b52d7cb-174e-4522-81d5-94bf3fbffb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:05:18.920272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 578/1175] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:05:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:05:18.945835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8a261-070e-4356-ad6f-e05f270771ad None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:19.056263 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8a261-070e-4356-ad6f-e05f270771ad None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:19.090895 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eac700-53c3-4809-970c-7cbaff5de14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:19.091781 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eac700-53c3-4809-970c-7cbaff5de14b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:19.092060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eac700-53c3-4809-970c-7cbaff5de14b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:19.092337 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eac700-53c3-4809-970c-7cbaff5de14b None admin] PATH_INFO: `/v3/domains/91a46aa676bc445fa166f8561029562d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:19.093801 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eac700-53c3-4809-970c-7cbaff5de14b None admin] RBAC: Authorizing `identity:get_domain(domain_id=91a46aa676bc445fa166f8561029562d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:19.095236 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eac700-53c3-4809-970c-7cbaff5de14b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:19.097035 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 598/1176] 10.222.0.27 () {56 vars in 1183 bytes} [Mon Jun 27 21:05:18 2022] GET /identity/v3/domains/91a46aa676bc445fa166f8561029562d => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:19.104136 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52d7cb-174e-4522-81d5-94bf3fbffb97 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:19.221884 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52d7cb-174e-4522-81d5-94bf3fbffb97 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:19.252458 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:19.259355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd74ab-5993-48be-abcb-d14af2efc614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:19.260065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd74ab-5993-48be-abcb-d14af2efc614 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:19.260344 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd74ab-5993-48be-abcb-d14af2efc614 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:19.260625 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd74ab-5993-48be-abcb-d14af2efc614 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:19.261337 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd74ab-5993-48be-abcb-d14af2efc614 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:19.262801 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd74ab-5993-48be-abcb-d14af2efc614 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:19.284751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 579/1177] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Jun 27 21:05:19 2022] POST /identity/v3/users => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:05:21.323269 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc687f9d-e3e5-4488-aa2c-a639a87ecae3 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:21.323872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc687f9d-e3e5-4488-aa2c-a639a87ecae3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:21.323872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc687f9d-e3e5-4488-aa2c-a639a87ecae3 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:21.325245 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 599/1178] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:05:21.331854 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0dbdf-efe3-46f3-bf72-f6f5659a46d1 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:21.332577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0dbdf-efe3-46f3-bf72-f6f5659a46d1 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:21.332737 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0dbdf-efe3-46f3-bf72-f6f5659a46d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:21.353329 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5fd0dbdf-efe3-46f3-bf72-f6f5659a46d1 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:05:21.354522 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5fd0dbdf-efe3-46f3-bf72-f6f5659a46d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:05:21.425861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 580/1179] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:05:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:05:21.449721 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc687f9d-e3e5-4488-aa2c-a639a87ecae3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:21.496915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:21.563656 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc687f9d-e3e5-4488-aa2c-a639a87ecae3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:21.598208 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245956b3-47c9-4bdc-abea-9aaa6bfbb2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:21.598931 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245956b3-47c9-4bdc-abea-9aaa6bfbb2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:21.599208 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245956b3-47c9-4bdc-abea-9aaa6bfbb2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:21.599474 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245956b3-47c9-4bdc-abea-9aaa6bfbb2e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:21.610461 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245956b3-47c9-4bdc-abea-9aaa6bfbb2e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:05:21.618088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-245956b3-47c9-4bdc-abea-9aaa6bfbb2e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:05:21.618088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:05:21.618088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:05:21.618088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:05:21.618088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:05:21.618088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:05:21.618088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:05:21.618088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:05:21.618088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:05:21.618088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:05:21.618088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:05:21.618088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:05:21.618088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:05:21.618088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:05:21.618088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:05:21.618088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:05:21.619179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:05:21.619179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:05:21.619179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:05:21.619179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:05:21.619179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:05:21.619179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:05:21.619179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:05:21.619179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:05:21.619179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:05:21.619179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:05:21.619179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:05:21.619179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:05:21.619179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:05:21.619179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:05:21.619179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:05:21.619179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:05:21.620354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:05:21.620354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:05:21.620354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:05:21.620354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:05:21.620354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:05:21.620354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:05:21.620354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:05:21.620354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:05:21.620354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:05:21.620354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:05:21.620354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:05:21.620354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:05:21.620354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 600/1180] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:05:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:05:21.628100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0dbdf-efe3-46f3-bf72-f6f5659a46d1 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:21.728723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0dbdf-efe3-46f3-bf72-f6f5659a46d1 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:21.746029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:21.770610 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa087e-588d-4959-aa59-875f9f3df4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:21.771295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa087e-588d-4959-aa59-875f9f3df4aa None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:21.771577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa087e-588d-4959-aa59-875f9f3df4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:21.771892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa087e-588d-4959-aa59-875f9f3df4aa None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:21.772649 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa087e-588d-4959-aa59-875f9f3df4aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:05:21.772959 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa087e-588d-4959-aa59-875f9f3df4aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:21.774349 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa087e-588d-4959-aa59-875f9f3df4aa None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:21.783095 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 581/1181] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:05:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:05:21.789007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245956b3-47c9-4bdc-abea-9aaa6bfbb2e8 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:21.899464 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245956b3-47c9-4bdc-abea-9aaa6bfbb2e8 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:21.901004 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:21.936249 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673eb007-0f5f-4ecd-bd24-588b210809bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:21.937120 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673eb007-0f5f-4ecd-bd24-588b210809bd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:21.937394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673eb007-0f5f-4ecd-bd24-588b210809bd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:21.937662 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673eb007-0f5f-4ecd-bd24-588b210809bd None admin] PATH_INFO: `/v3/users/6523f50e283447d9b87e6bbd096f60e8` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:21.951076 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673eb007-0f5f-4ecd-bd24-588b210809bd None admin] RBAC: Authorizing `identity:get_user(user_id=6523f50e283447d9b87e6bbd096f60e8)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:21.952552 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673eb007-0f5f-4ecd-bd24-588b210809bd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:21.954169 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 601/1182] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:05:21 2022] GET /identity/v3/users/6523f50e283447d9b87e6bbd096f60e8 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:21.960077 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa087e-588d-4959-aa59-875f9f3df4aa None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:22.065855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa087e-588d-4959-aa59-875f9f3df4aa None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:22.102214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bd9bb-5e19-4032-a742-f667adf39e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:22.102851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bd9bb-5e19-4032-a742-f667adf39e0e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:22.103126 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bd9bb-5e19-4032-a742-f667adf39e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:22.103397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bd9bb-5e19-4032-a742-f667adf39e0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:22.151975 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:22.185388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bd9bb-5e19-4032-a742-f667adf39e0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:22.186861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bd9bb-5e19-4032-a742-f667adf39e0e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:22.292599 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 582/1183] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:05:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:05:22.302310 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673eb007-0f5f-4ecd-bd24-588b210809bd None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:22.412891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673eb007-0f5f-4ecd-bd24-588b210809bd None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:22.454699 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d5416-4e51-4151-afe7-895c4fa5f895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:22.455574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d5416-4e51-4151-afe7-895c4fa5f895 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:22.455889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d5416-4e51-4151-afe7-895c4fa5f895 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:22.456158 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d5416-4e51-4151-afe7-895c4fa5f895 None admin] PATH_INFO: `/v3/projects/4975b4a4d7dc4526aa420c3f44cf1307` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:22.457471 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d5416-4e51-4151-afe7-895c4fa5f895 None admin] RBAC: Authorizing `identity:get_project(project_id=4975b4a4d7dc4526aa420c3f44cf1307)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:22.458930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d5416-4e51-4151-afe7-895c4fa5f895 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:22.460645 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 602/1184] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:05:22 2022] GET /identity/v3/projects/4975b4a4d7dc4526aa420c3f44cf1307 => generated 361 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:22.466979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bd9bb-5e19-4032-a742-f667adf39e0e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:22.554773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:22.578264 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bd9bb-5e19-4032-a742-f667adf39e0e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:22.616809 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be780c-5c22-4246-bc20-23577f8571a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:22.617465 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be780c-5c22-4246-bc20-23577f8571a1 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:22.617751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be780c-5c22-4246-bc20-23577f8571a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:22.618027 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be780c-5c22-4246-bc20-23577f8571a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:22.618865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be780c-5c22-4246-bc20-23577f8571a1 None admin] RBAC: Adding query filter params (role.id=5c3c095bb53943eca95c4e049cd8cbf5, scope.project.id=4975b4a4d7dc4526aa420c3f44cf1307, user.id=6523f50e283447d9b87e6bbd096f60e8) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:05:22.619180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be780c-5c22-4246-bc20-23577f8571a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:22.620666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be780c-5c22-4246-bc20-23577f8571a1 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:22.626376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 583/1185] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:05:22 2022] GET /identity/v3/role_assignments?user.id=6523f50e283447d9b87e6bbd096f60e8&scope.project.id=4975b4a4d7dc4526aa420c3f44cf1307&role.id=5c3c095bb53943eca95c4e049cd8cbf5 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:24.611679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9841a-862d-4d0c-aa28-39ffcd346b04 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:24.612563 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9841a-862d-4d0c-aa28-39ffcd346b04 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:24.612563 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9841a-862d-4d0c-aa28-39ffcd346b04 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:24.614798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 603/1186] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:05:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:05:24.623864 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa524ef-8ce9-4396-90f3-1b094343521c None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:24.624301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa524ef-8ce9-4396-90f3-1b094343521c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:24.624683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa524ef-8ce9-4396-90f3-1b094343521c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:24.646745 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-cfa524ef-8ce9-4396-90f3-1b094343521c None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:05:24.647968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-cfa524ef-8ce9-4396-90f3-1b094343521c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:05:24.720682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 584/1187] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:05:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:05:24.747978 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9841a-862d-4d0c-aa28-39ffcd346b04 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:24.780598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:24.859601 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9841a-862d-4d0c-aa28-39ffcd346b04 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:24.895669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4dcb3-c5fa-465c-b546-cca2a9deb11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:24.896409 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4dcb3-c5fa-465c-b546-cca2a9deb11e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:24.896692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4dcb3-c5fa-465c-b546-cca2a9deb11e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:24.896964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4dcb3-c5fa-465c-b546-cca2a9deb11e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:24.907995 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4dcb3-c5fa-465c-b546-cca2a9deb11e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:05:24.915635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-c3d4dcb3-c5fa-465c-b546-cca2a9deb11e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:05:24.915635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:05:24.915635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:05:24.915635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:05:24.915635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:05:24.915635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:05:24.915635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:05:24.915635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:05:24.915635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:05:24.915635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:05:24.915635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:05:24.915635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:05:24.915635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:05:24.915635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:05:24.915635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:05:24.915635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:05:24.916676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:05:24.916676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:05:24.916676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:05:24.916676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:05:24.916676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:05:24.916676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:05:24.916676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:05:24.916676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:05:24.916676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:05:24.916676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:05:24.916676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:05:24.916676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:05:24.916676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:05:24.916676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:05:24.916676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:05:24.916676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:05:24.917683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:05:24.917683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:05:24.917683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:05:24.917683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:05:24.917683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:05:24.917683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:05:24.917683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:05:24.917683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:05:24.917683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:05:24.917683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:05:24.917683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:05:24.917683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:05:24.919780 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 604/1188] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:05:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:05:24.923797 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa524ef-8ce9-4396-90f3-1b094343521c None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:25.034492 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:25.062207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa524ef-8ce9-4396-90f3-1b094343521c None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:25.097180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1971e82-5cb6-4509-bfe7-37406ffd2e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:25.097840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1971e82-5cb6-4509-bfe7-37406ffd2e05 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:25.098120 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1971e82-5cb6-4509-bfe7-37406ffd2e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:25.098388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1971e82-5cb6-4509-bfe7-37406ffd2e05 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:25.099130 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1971e82-5cb6-4509-bfe7-37406ffd2e05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:05:25.099440 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1971e82-5cb6-4509-bfe7-37406ffd2e05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:25.100860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1971e82-5cb6-4509-bfe7-37406ffd2e05 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:25.109018 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 585/1189] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:05:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:25.117780 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4dcb3-c5fa-465c-b546-cca2a9deb11e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:25.207947 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:25.221052 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4dcb3-c5fa-465c-b546-cca2a9deb11e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:25.255498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f54f63b-fe5a-4bc7-93ac-4b92411c39e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:25.256392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54f63b-fe5a-4bc7-93ac-4b92411c39e8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:25.256666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54f63b-fe5a-4bc7-93ac-4b92411c39e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:25.256931 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54f63b-fe5a-4bc7-93ac-4b92411c39e8 None admin] PATH_INFO: `/v3/users/6523f50e283447d9b87e6bbd096f60e8` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:25.258154 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f54f63b-fe5a-4bc7-93ac-4b92411c39e8 None admin] RBAC: Authorizing `identity:get_user(user_id=6523f50e283447d9b87e6bbd096f60e8)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:25.259633 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f54f63b-fe5a-4bc7-93ac-4b92411c39e8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:25.261234 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 605/1190] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:05:25 2022] GET /identity/v3/users/6523f50e283447d9b87e6bbd096f60e8 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:25.267832 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1971e82-5cb6-4509-bfe7-37406ffd2e05 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:25.377033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1971e82-5cb6-4509-bfe7-37406ffd2e05 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:25.412271 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c446e3a-4e66-4400-9f69-a792e168ff6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:25.413132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c446e3a-4e66-4400-9f69-a792e168ff6e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:25.413410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c446e3a-4e66-4400-9f69-a792e168ff6e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:25.413680 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c446e3a-4e66-4400-9f69-a792e168ff6e None admin] PATH_INFO: `/v3/projects/4975b4a4d7dc4526aa420c3f44cf1307` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:25.414974 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c446e3a-4e66-4400-9f69-a792e168ff6e None admin] RBAC: Authorizing `identity:get_project(project_id=4975b4a4d7dc4526aa420c3f44cf1307)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:25.416451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c446e3a-4e66-4400-9f69-a792e168ff6e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:25.418112 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 586/1191] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:05:25 2022] GET /identity/v3/projects/4975b4a4d7dc4526aa420c3f44cf1307 => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:25.424937 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f54f63b-fe5a-4bc7-93ac-4b92411c39e8 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:25.542021 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f54f63b-fe5a-4bc7-93ac-4b92411c39e8 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:25.580354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4befb-4d3e-423b-92c9-0ef028099a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:25.581237 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4befb-4d3e-423b-92c9-0ef028099a94 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:25.581512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4befb-4d3e-423b-92c9-0ef028099a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:25.581779 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4befb-4d3e-423b-92c9-0ef028099a94 None admin] PATH_INFO: `/v3/projects/4975b4a4d7dc4526aa420c3f44cf1307/users/6523f50e283447d9b87e6bbd096f60e8/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:25.583987 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4befb-4d3e-423b-92c9-0ef028099a94 None admin] RBAC: Authorizing `identity:create_grant(project_id=4975b4a4d7dc4526aa420c3f44cf1307, user_id=6523f50e283447d9b87e6bbd096f60e8, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:25.585430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4befb-4d3e-423b-92c9-0ef028099a94 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:25.600807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 606/1192] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:05:25 2022] PUT /identity/v3/projects/4975b4a4d7dc4526aa420c3f44cf1307/users/6523f50e283447d9b87e6bbd096f60e8/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:05:27.573520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b30a96-61d4-4821-9d8f-67407409261d None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:27.574420 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b30a96-61d4-4821-9d8f-67407409261d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:27.574550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b30a96-61d4-4821-9d8f-67407409261d None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:27.576566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 587/1193] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:05:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:05:27.583361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5889839-7c81-409d-b996-cbc7585cff57 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:27.583811 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5889839-7c81-409d-b996-cbc7585cff57 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:27.583964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5889839-7c81-409d-b996-cbc7585cff57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:27.610062 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a5889839-7c81-409d-b996-cbc7585cff57 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:05:27.611257 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a5889839-7c81-409d-b996-cbc7585cff57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:05:27.622257 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:27.681208 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 607/1194] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:05:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:05:27.705091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b30a96-61d4-4821-9d8f-67407409261d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:27.710776 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:27.829129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b30a96-61d4-4821-9d8f-67407409261d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:27.864435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df6ad9-ede7-4d1b-9bae-29efab222273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:27.865225 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df6ad9-ede7-4d1b-9bae-29efab222273 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:27.865501 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df6ad9-ede7-4d1b-9bae-29efab222273 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:27.865784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df6ad9-ede7-4d1b-9bae-29efab222273 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:27.876994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df6ad9-ede7-4d1b-9bae-29efab222273 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:05:27.884692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-41df6ad9-ede7-4d1b-9bae-29efab222273 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:05:27.884692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:05:27.884692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:05:27.884692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:05:27.884692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:05:27.884692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:05:27.884692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:05:27.884692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:05:27.884692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:05:27.884692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:05:27.884692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:05:27.884692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:05:27.884692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:05:27.884692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:05:27.884692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:05:27.884692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:05:27.885860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:05:27.885860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:05:27.885860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:05:27.885860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:05:27.885860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:05:27.885860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:05:27.885860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:05:27.885860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:05:27.885860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:05:27.885860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:05:27.885860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:05:27.885860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:05:27.885860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:05:27.885860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:05:27.885860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:05:27.885860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:05:27.887063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:05:27.887063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:05:27.887063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:05:27.887063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:05:27.887063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:05:27.887063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:05:27.887063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:05:27.887063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:05:27.887063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:05:27.887063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:05:27.887063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:05:27.887063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:05:27.887063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 588/1195] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:05:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:05:27.892703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5889839-7c81-409d-b996-cbc7585cff57 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:28.000573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5889839-7c81-409d-b996-cbc7585cff57 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:28.044228 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0015a9-d327-4585-bad0-be719f43d70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:28.045261 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0015a9-d327-4585-bad0-be719f43d70e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:28.045689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0015a9-d327-4585-bad0-be719f43d70e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:28.046109 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0015a9-d327-4585-bad0-be719f43d70e None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:28.047304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0015a9-d327-4585-bad0-be719f43d70e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:05:28.047821 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0015a9-d327-4585-bad0-be719f43d70e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:28.050041 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0015a9-d327-4585-bad0-be719f43d70e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:28.063039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 608/1196] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:05:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:05:28.071636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df6ad9-ede7-4d1b-9bae-29efab222273 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:28.139050 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:28.178371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df6ad9-ede7-4d1b-9bae-29efab222273 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:28.213403 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3705da-5441-4ba5-8d0a-dd0289c91ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:28.214271 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3705da-5441-4ba5-8d0a-dd0289c91ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:28.214549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3705da-5441-4ba5-8d0a-dd0289c91ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:28.214826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3705da-5441-4ba5-8d0a-dd0289c91ff7 None admin] PATH_INFO: `/v3/users/6523f50e283447d9b87e6bbd096f60e8` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:28.216233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3705da-5441-4ba5-8d0a-dd0289c91ff7 None admin] RBAC: Authorizing `identity:get_user(user_id=6523f50e283447d9b87e6bbd096f60e8)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:28.217680 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3705da-5441-4ba5-8d0a-dd0289c91ff7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:28.219272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 589/1197] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Jun 27 21:05:28 2022] GET /identity/v3/users/6523f50e283447d9b87e6bbd096f60e8 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:28.225268 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0015a9-d327-4585-bad0-be719f43d70e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:28.237343 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:28.331200 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0015a9-d327-4585-bad0-be719f43d70e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:28.368804 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76efdefb-3068-4dae-87d5-c4a6748e47bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:28.369448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efdefb-3068-4dae-87d5-c4a6748e47bf None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:28.369735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efdefb-3068-4dae-87d5-c4a6748e47bf None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:28.370014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efdefb-3068-4dae-87d5-c4a6748e47bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:28.439360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76efdefb-3068-4dae-87d5-c4a6748e47bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:28.440793 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76efdefb-3068-4dae-87d5-c4a6748e47bf None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:28.451127 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:28.543897 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 609/1198] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:05:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:05:28.550039 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3705da-5441-4ba5-8d0a-dd0289c91ff7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:28.665300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3705da-5441-4ba5-8d0a-dd0289c91ff7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:28.697792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:28.704331 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40bef4-2478-49e0-9a20-044cf70adafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:28.705195 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40bef4-2478-49e0-9a20-044cf70adafc None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:28.705471 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40bef4-2478-49e0-9a20-044cf70adafc None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:28.705742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40bef4-2478-49e0-9a20-044cf70adafc None admin] PATH_INFO: `/v3/projects/4975b4a4d7dc4526aa420c3f44cf1307` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:28.707038 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a40bef4-2478-49e0-9a20-044cf70adafc None admin] RBAC: Authorizing `identity:get_project(project_id=4975b4a4d7dc4526aa420c3f44cf1307)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:28.708534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a40bef4-2478-49e0-9a20-044cf70adafc None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:28.710227 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 590/1199] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Jun 27 21:05:28 2022] GET /identity/v3/projects/4975b4a4d7dc4526aa420c3f44cf1307 => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:28.718994 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76efdefb-3068-4dae-87d5-c4a6748e47bf None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:28.825746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76efdefb-3068-4dae-87d5-c4a6748e47bf None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:28.834897 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:28.864852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99dd0ac-27c1-4a8e-99bb-d26348da326a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:28.865506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99dd0ac-27c1-4a8e-99bb-d26348da326a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:28.865793 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99dd0ac-27c1-4a8e-99bb-d26348da326a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:28.866072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99dd0ac-27c1-4a8e-99bb-d26348da326a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:28.866901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99dd0ac-27c1-4a8e-99bb-d26348da326a None admin] RBAC: Adding query filter params (role.id=5c3c095bb53943eca95c4e049cd8cbf5, scope.project.id=4975b4a4d7dc4526aa420c3f44cf1307, user.id=6523f50e283447d9b87e6bbd096f60e8) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:05:28.867213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99dd0ac-27c1-4a8e-99bb-d26348da326a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:28.868710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99dd0ac-27c1-4a8e-99bb-d26348da326a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:28.875190 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 610/1200] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:05:28 2022] GET /identity/v3/role_assignments?user.id=6523f50e283447d9b87e6bbd096f60e8&scope.project.id=4975b4a4d7dc4526aa420c3f44cf1307&role.id=5c3c095bb53943eca95c4e049cd8cbf5 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:38.872085 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=14 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:24.022823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff382c28-2263-4367-a1a6-f6c3ef1ba254 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:24.022823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff382c28-2263-4367-a1a6-f6c3ef1ba254 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:24.022823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff382c28-2263-4367-a1a6-f6c3ef1ba254 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:24.022823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 591/1201] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:06:24.029519 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1a2ca-bdb0-4db7-adf2-a1a51a1031e0 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:24.029988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1a2ca-bdb0-4db7-adf2-a1a51a1031e0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:24.030271 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1a2ca-bdb0-4db7-adf2-a1a51a1031e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:24.048518 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e2e1a2ca-bdb0-4db7-adf2-a1a51a1031e0 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:06:24.049901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e2e1a2ca-bdb0-4db7-adf2-a1a51a1031e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:06:24.134546 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 611/1202] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:06:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:06:24.161216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff382c28-2263-4367-a1a6-f6c3ef1ba254 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:24.306080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff382c28-2263-4367-a1a6-f6c3ef1ba254 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:24.350532 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d35b2-4231-4f5c-a1ac-4b3ca2ae2c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:24.351375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d35b2-4231-4f5c-a1ac-4b3ca2ae2c0b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:24.351655 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d35b2-4231-4f5c-a1ac-4b3ca2ae2c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:24.352081 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d35b2-4231-4f5c-a1ac-4b3ca2ae2c0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:24.365081 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d35b2-4231-4f5c-a1ac-4b3ca2ae2c0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:24.366653 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d35b2-4231-4f5c-a1ac-4b3ca2ae2c0b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:24.386986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-297d35b2-4231-4f5c-a1ac-4b3ca2ae2c0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:24.386986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:24.386986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:24.386986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:24.386986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:24.386986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:24.386986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:24.386986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:24.386986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:24.386986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:24.386986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:24.386986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:24.386986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:24.386986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:24.386986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:24.388613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:24.388613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:24.388613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:24.388613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:24.388613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:24.388613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:24.388613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:24.388613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:24.388613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:24.388613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:24.388613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:24.388613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:24.388613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:24.388613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:24.388613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:24.388613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:24.388613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:24.389590 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:24.389590 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:24.389590 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:24.389590 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:06:24.389590 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 592/1203] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:24.395893 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1a2ca-bdb0-4db7-adf2-a1a51a1031e0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:24.538600 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1a2ca-bdb0-4db7-adf2-a1a51a1031e0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:24.574780 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:24.584543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68509b8-f806-4bd2-9b03-36381bce1ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:24.585303 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68509b8-f806-4bd2-9b03-36381bce1ffc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:24.585588 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68509b8-f806-4bd2-9b03-36381bce1ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:24.585865 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68509b8-f806-4bd2-9b03-36381bce1ffc None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:24.586615 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68509b8-f806-4bd2-9b03-36381bce1ffc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:24.586934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68509b8-f806-4bd2-9b03-36381bce1ffc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:24.588608 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68509b8-f806-4bd2-9b03-36381bce1ffc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:24.602783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 612/1204] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:24.609314 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d35b2-4231-4f5c-a1ac-4b3ca2ae2c0b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:24.683390 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:24.746057 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d35b2-4231-4f5c-a1ac-4b3ca2ae2c0b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:24.793131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b1064-782d-4b42-932b-dba127268654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:24.793951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b1064-782d-4b42-932b-dba127268654 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:24.794262 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b1064-782d-4b42-932b-dba127268654 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:24.794589 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b1064-782d-4b42-932b-dba127268654 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:24.796087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b1064-782d-4b42-932b-dba127268654 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:24.797719 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b1064-782d-4b42-932b-dba127268654 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:24.831127 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 593/1205] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Jun 27 21:06:24 2022] POST /identity/v3/users => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:06:27.205327 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03187dc6-2531-44e3-aef3-75928b73932d None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:27.205891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03187dc6-2531-44e3-aef3-75928b73932d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:27.205891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03187dc6-2531-44e3-aef3-75928b73932d None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:27.207249 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 613/1206] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:06:27.213254 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e46ac9-28ed-40cd-85a5-11aeb342780e None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:27.213694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e46ac9-28ed-40cd-85a5-11aeb342780e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:27.213783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e46ac9-28ed-40cd-85a5-11aeb342780e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:27.234171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e2e46ac9-28ed-40cd-85a5-11aeb342780e None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:06:27.235572 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e2e46ac9-28ed-40cd-85a5-11aeb342780e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:06:27.319379 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 594/1207] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:06:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:06:27.343282 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03187dc6-2531-44e3-aef3-75928b73932d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:27.465602 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03187dc6-2531-44e3-aef3-75928b73932d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:27.502513 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb107f6-b3cd-452c-88f9-01f149947eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:27.503474 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb107f6-b3cd-452c-88f9-01f149947eec None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:27.503840 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb107f6-b3cd-452c-88f9-01f149947eec None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:27.504188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb107f6-b3cd-452c-88f9-01f149947eec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:27.517375 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb107f6-b3cd-452c-88f9-01f149947eec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:06:27.526259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-2eb107f6-b3cd-452c-88f9-01f149947eec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:06:27.526259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:27.526259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:27.526259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:27.526259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:27.526259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:27.526259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:27.526259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:27.526259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:27.526259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:27.526259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:27.526259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:27.526259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:06:27.526259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:06:27.526259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:06:27.526259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:06:27.527436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:06:27.527436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:06:27.527436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:06:27.527436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:06:27.527436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:27.527436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:27.527436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:27.527436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:27.527436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:27.527436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:27.527436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:27.527436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:27.527436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:27.527436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:27.527436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:27.527436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:27.534128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:27.534128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:27.534128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:27.534128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:27.534128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:06:27.534128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:06:27.534128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:06:27.534128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:06:27.534128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:06:27.534128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:06:27.534128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:06:27.534128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:27.534128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 614/1208] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:06:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:27.535137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e46ac9-28ed-40cd-85a5-11aeb342780e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:27.583594 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:27.659076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e46ac9-28ed-40cd-85a5-11aeb342780e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:27.701120 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8a259-2970-4e63-b4d8-19d5d6765cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:27.701922 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8a259-2970-4e63-b4d8-19d5d6765cfc None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:27.702308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8a259-2970-4e63-b4d8-19d5d6765cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:27.702996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8a259-2970-4e63-b4d8-19d5d6765cfc None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:27.703955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8a259-2970-4e63-b4d8-19d5d6765cfc None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:27.704363 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8a259-2970-4e63-b4d8-19d5d6765cfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:27.705940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8a259-2970-4e63-b4d8-19d5d6765cfc None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:27.716346 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 595/1209] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:06:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:06:27.722338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb107f6-b3cd-452c-88f9-01f149947eec None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:27.780879 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:27.845465 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb107f6-b3cd-452c-88f9-01f149947eec None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:27.885622 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af0e06-e1fc-4866-b796-0ff01a8a1359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:27.886828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af0e06-e1fc-4866-b796-0ff01a8a1359 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:27.887244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af0e06-e1fc-4866-b796-0ff01a8a1359 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:27.887593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af0e06-e1fc-4866-b796-0ff01a8a1359 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:27.901390 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af0e06-e1fc-4866-b796-0ff01a8a1359 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:27.903266 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af0e06-e1fc-4866-b796-0ff01a8a1359 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:27.925708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-54af0e06-e1fc-4866-b796-0ff01a8a1359 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:27.925708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:27.925708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:27.925708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:27.925708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:27.925708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:27.925708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:27.925708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:27.925708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:27.925708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:27.925708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:27.925708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:27.925708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:27.925708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:27.925708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:27.926852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:27.926852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:27.926852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:27.926852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:27.926852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:27.926852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:27.926852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:27.926852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:27.926852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:27.926852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:27.926852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:27.926852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:27.926852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:27.926852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:27.926852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:27.926852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:27.926852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:27.928095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:27.928095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:27.928095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:27.928095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:27.928095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 615/1210] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:27.933165 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8a259-2970-4e63-b4d8-19d5d6765cfc None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:28.070327 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:28.073703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8a259-2970-4e63-b4d8-19d5d6765cfc None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:28.113811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338098ab-a748-4276-8b2a-b505d1fcd62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:28.114655 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338098ab-a748-4276-8b2a-b505d1fcd62a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:28.114950 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338098ab-a748-4276-8b2a-b505d1fcd62a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:28.115273 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338098ab-a748-4276-8b2a-b505d1fcd62a None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:28.116132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338098ab-a748-4276-8b2a-b505d1fcd62a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:28.116446 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338098ab-a748-4276-8b2a-b505d1fcd62a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:28.118011 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338098ab-a748-4276-8b2a-b505d1fcd62a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:28.131398 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 596/1211] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:28.138349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af0e06-e1fc-4866-b796-0ff01a8a1359 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:28.249264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af0e06-e1fc-4866-b796-0ff01a8a1359 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:28.250833 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:28.285893 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df9c02-9951-4700-882a-dffb73d3be4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:28.286761 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df9c02-9951-4700-882a-dffb73d3be4b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:28.286997 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df9c02-9951-4700-882a-dffb73d3be4b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:28.287246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df9c02-9951-4700-882a-dffb73d3be4b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:28.294775 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df9c02-9951-4700-882a-dffb73d3be4b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:28.296293 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df9c02-9951-4700-882a-dffb73d3be4b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:28.311835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-d3df9c02-9951-4700-882a-dffb73d3be4b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:28.311835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:28.311835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:28.311835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:28.311835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:28.311835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:28.311835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:28.311835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:28.311835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:28.311835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:28.311835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:28.311835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:28.311835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:06:28.311835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:06:28.311835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:06:28.311835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:06:28.313014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:28.313014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:28.313014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:06:28.313014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:28.313014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:06:28.313014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:28.313014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:28.313014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:28.313014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:28.313014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:28.313014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:28.313014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:28.313014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:28.313014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:28.313014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:28.313014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:28.313014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:28.313014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:28.314151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:28.314151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:28.314151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:06:28.314151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:06:28.314151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:06:28.314151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:06:28.314151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:06:28.314151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:06:28.314151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:28.314151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:28.314151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 616/1212] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:06:28.319190 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338098ab-a748-4276-8b2a-b505d1fcd62a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:28.439356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338098ab-a748-4276-8b2a-b505d1fcd62a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:28.476898 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3de11-6489-45e1-8a51-d3c19fe45cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:28.477578 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3de11-6489-45e1-8a51-d3c19fe45cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:28.477859 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3de11-6489-45e1-8a51-d3c19fe45cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:28.478133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3de11-6489-45e1-8a51-d3c19fe45cb3 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:28.478977 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3de11-6489-45e1-8a51-d3c19fe45cb3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:28.479277 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3de11-6489-45e1-8a51-d3c19fe45cb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:28.480835 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3de11-6489-45e1-8a51-d3c19fe45cb3 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:28.504095 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 597/1213] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Jun 27 21:06:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:28.505964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df9c02-9951-4700-882a-dffb73d3be4b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:28.620078 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df9c02-9951-4700-882a-dffb73d3be4b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:28.658182 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0507b-053b-4b17-bf7d-acbdabc5e38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:28.658969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0507b-053b-4b17-bf7d-acbdabc5e38d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:28.659248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0507b-053b-4b17-bf7d-acbdabc5e38d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:28.659549 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0507b-053b-4b17-bf7d-acbdabc5e38d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:28.692596 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:28.743314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df0507b-053b-4b17-bf7d-acbdabc5e38d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:28.744836 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df0507b-053b-4b17-bf7d-acbdabc5e38d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:28.855484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 617/1214] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:06:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:06:28.861428 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3de11-6489-45e1-8a51-d3c19fe45cb3 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:28.883040 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:28.990789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3de11-6489-45e1-8a51-d3c19fe45cb3 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:29.041997 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0172e9-91f2-42fb-ac2c-68e6cd38ee37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:29.042909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0172e9-91f2-42fb-ac2c-68e6cd38ee37 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:29.043189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0172e9-91f2-42fb-ac2c-68e6cd38ee37 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:29.043463 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0172e9-91f2-42fb-ac2c-68e6cd38ee37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:29.054661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0172e9-91f2-42fb-ac2c-68e6cd38ee37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:29.056238 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0172e9-91f2-42fb-ac2c-68e6cd38ee37 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:29.075189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-9b0172e9-91f2-42fb-ac2c-68e6cd38ee37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:29.075189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:29.075189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:29.075189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:29.075189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:29.075189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:29.075189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:29.075189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:29.075189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:29.075189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:29.075189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:29.075189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:29.075189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:29.075189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:29.075189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:29.076371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:29.076371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:29.076371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:29.076371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:29.076371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:29.076371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:29.076371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:29.076371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:29.076371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:29.076371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:29.076371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:29.076371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:29.076371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:29.076371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:29.076371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:29.076371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:29.076371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:29.077302 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:29.077302 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:29.077302 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:29.077302 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:06:29.077302 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 598/1215] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:06:29.082673 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0507b-053b-4b17-bf7d-acbdabc5e38d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:29.163698 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:29.217432 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0507b-053b-4b17-bf7d-acbdabc5e38d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:29.251902 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1e97a-a85c-4fe0-a743-d6855ecde833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:29.252541 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1e97a-a85c-4fe0-a743-d6855ecde833 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:29.252773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1e97a-a85c-4fe0-a743-d6855ecde833 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:29.253028 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1e97a-a85c-4fe0-a743-d6855ecde833 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:29.253742 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1e97a-a85c-4fe0-a743-d6855ecde833 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:29.254046 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1e97a-a85c-4fe0-a743-d6855ecde833 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:29.255429 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1e97a-a85c-4fe0-a743-d6855ecde833 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:29.271658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 618/1216] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:06:29.272996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0172e9-91f2-42fb-ac2c-68e6cd38ee37 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:29.355556 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:29.389941 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0172e9-91f2-42fb-ac2c-68e6cd38ee37 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:29.430745 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10653c29-2120-466d-8d8c-287f2b8bf0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:29.431902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10653c29-2120-466d-8d8c-287f2b8bf0bb None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:29.432176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10653c29-2120-466d-8d8c-287f2b8bf0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:29.432434 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10653c29-2120-466d-8d8c-287f2b8bf0bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:29.440124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10653c29-2120-466d-8d8c-287f2b8bf0bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:29.441594 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10653c29-2120-466d-8d8c-287f2b8bf0bb None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:29.455577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-10653c29-2120-466d-8d8c-287f2b8bf0bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:29.455577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:29.455577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:29.455577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:29.455577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:29.455577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:29.455577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:29.455577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:29.455577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:29.455577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:29.455577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:29.455577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:29.455577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:06:29.455577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:06:29.455577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:06:29.456587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:06:29.456587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:29.456587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:29.456587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:29.456587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:29.456587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:29.456587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:29.456587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:29.456587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:29.456587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:29.456587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:29.456587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:29.456587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:29.456587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:29.456587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:29.456587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:29.456587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:29.458148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:06:29.458148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:06:29.458148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:06:29.458148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:06:29.458148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:06:29.458148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:06:29.458148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:29.458148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:06:29.458148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 599/1217] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:06:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:06:29.462632 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1e97a-a85c-4fe0-a743-d6855ecde833 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:29.581061 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1e97a-a85c-4fe0-a743-d6855ecde833 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:29.607358 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:29.621880 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fe6dd-a9f5-4244-8be5-665ab014002f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:29.622608 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fe6dd-a9f5-4244-8be5-665ab014002f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:29.622936 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fe6dd-a9f5-4244-8be5-665ab014002f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:29.623164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fe6dd-a9f5-4244-8be5-665ab014002f None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:29.624078 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fe6dd-a9f5-4244-8be5-665ab014002f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:29.624452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fe6dd-a9f5-4244-8be5-665ab014002f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:29.626079 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fe6dd-a9f5-4244-8be5-665ab014002f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:29.638922 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 619/1218] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:06:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:29.644910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10653c29-2120-466d-8d8c-287f2b8bf0bb None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:29.769026 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10653c29-2120-466d-8d8c-287f2b8bf0bb None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:29.812934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:29.819956 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e2b2f-a5bd-4fb7-9b7b-3612957839f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:29.820845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e2b2f-a5bd-4fb7-9b7b-3612957839f6 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:29.821300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e2b2f-a5bd-4fb7-9b7b-3612957839f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:29.821576 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e2b2f-a5bd-4fb7-9b7b-3612957839f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:29.822415 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e2b2f-a5bd-4fb7-9b7b-3612957839f6 None admin] RBAC: Adding query filter params (role.id=fb120deba4284112b12de1a251b7da76, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=1f5fa9b0b0314ee9bb7e16b8464307b1) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:29.822728 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e2b2f-a5bd-4fb7-9b7b-3612957839f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:29.824278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e2b2f-a5bd-4fb7-9b7b-3612957839f6 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:29.836021 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 600/1219] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:06:29 2022] GET /identity/v3/role_assignments?user.id=1f5fa9b0b0314ee9bb7e16b8464307b1&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=fb120deba4284112b12de1a251b7da76 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:31.833913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a5a93-cef3-47fb-a430-f784cdb852d9 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:31.834486 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a5a93-cef3-47fb-a430-f784cdb852d9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:31.834486 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a5a93-cef3-47fb-a430-f784cdb852d9 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:31.836111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 620/1220] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:06:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:06:31.842955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a37790-ca0a-4170-8347-397149af5887 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:31.843424 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a37790-ca0a-4170-8347-397149af5887 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:31.843500 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a37790-ca0a-4170-8347-397149af5887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:31.860505 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-67a37790-ca0a-4170-8347-397149af5887 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:06:31.861694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-67a37790-ca0a-4170-8347-397149af5887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:06:31.931953 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 601/1221] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:06:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:06:31.957080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a5a93-cef3-47fb-a430-f784cdb852d9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:32.078897 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a5a93-cef3-47fb-a430-f784cdb852d9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:32.119997 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662e775-a1f9-49ac-a09e-377b65dc1104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:32.120747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662e775-a1f9-49ac-a09e-377b65dc1104 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:32.121028 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662e775-a1f9-49ac-a09e-377b65dc1104 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:32.121322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662e775-a1f9-49ac-a09e-377b65dc1104 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:32.131669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662e775-a1f9-49ac-a09e-377b65dc1104 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:06:32.139477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-9662e775-a1f9-49ac-a09e-377b65dc1104 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:06:32.139477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:32.139477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:32.139477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:32.139477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:32.139477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:32.139477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:32.139477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:32.139477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:32.139477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:32.139477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:32.139477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:32.139477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:06:32.139477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:06:32.139477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:06:32.139477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:06:32.140443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:06:32.140443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:06:32.140443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:06:32.140443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:06:32.140443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:32.140443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:32.140443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:32.140443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:32.140443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:32.140443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:32.140443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:32.140443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:32.140443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:32.140443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:32.140443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:32.140443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:32.141569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:32.141569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:32.141569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:32.141569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:32.141569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:06:32.141569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:06:32.141569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:06:32.141569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:06:32.141569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:06:32.141569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:06:32.141569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:06:32.141569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:32.141569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 621/1222] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:06:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:32.146963 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a37790-ca0a-4170-8347-397149af5887 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:32.248175 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a37790-ca0a-4170-8347-397149af5887 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:32.272551 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:32.284490 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93343a9-d764-427f-9f25-b7a473fe19b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:32.285168 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93343a9-d764-427f-9f25-b7a473fe19b8 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:32.285452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93343a9-d764-427f-9f25-b7a473fe19b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:32.285723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93343a9-d764-427f-9f25-b7a473fe19b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:32.286477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93343a9-d764-427f-9f25-b7a473fe19b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:32.286782 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93343a9-d764-427f-9f25-b7a473fe19b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:32.288192 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93343a9-d764-427f-9f25-b7a473fe19b8 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:32.296495 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 602/1223] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:06:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:32.302263 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662e775-a1f9-49ac-a09e-377b65dc1104 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:32.346433 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:32.410955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662e775-a1f9-49ac-a09e-377b65dc1104 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:32.445503 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b433db4-739e-4ee0-a3ed-ecf2ac367301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:32.446310 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b433db4-739e-4ee0-a3ed-ecf2ac367301 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:32.446588 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b433db4-739e-4ee0-a3ed-ecf2ac367301 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:32.446825 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b433db4-739e-4ee0-a3ed-ecf2ac367301 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:32.457639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b433db4-739e-4ee0-a3ed-ecf2ac367301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:32.459191 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b433db4-739e-4ee0-a3ed-ecf2ac367301 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:32.476322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-5b433db4-739e-4ee0-a3ed-ecf2ac367301 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:32.476322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:32.476322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:32.476322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:32.476322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:32.476322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:32.476322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:32.476322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:32.476322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:32.476322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:32.476322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:32.476322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:32.476322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:32.476322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:32.476322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:32.477294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:32.477294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:32.477294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:32.477294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:32.477294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:32.477294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:32.477294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:32.477294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:32.477294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:32.477294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:32.477294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:32.477294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:32.477294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:32.477294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:32.477294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:32.477294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:32.477294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:32.478516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:32.478516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:32.478516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:32.478516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:32.478516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 622/1224] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:32.483860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93343a9-d764-427f-9f25-b7a473fe19b8 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:32.582894 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93343a9-d764-427f-9f25-b7a473fe19b8 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:32.622641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b44e64-1f07-4dfa-a18a-8aaa05637058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:32.623326 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b44e64-1f07-4dfa-a18a-8aaa05637058 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:32.623621 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b44e64-1f07-4dfa-a18a-8aaa05637058 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:32.623920 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b44e64-1f07-4dfa-a18a-8aaa05637058 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:32.624655 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b44e64-1f07-4dfa-a18a-8aaa05637058 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:32.624965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b44e64-1f07-4dfa-a18a-8aaa05637058 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:32.626354 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b44e64-1f07-4dfa-a18a-8aaa05637058 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:32.639277 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 603/1225] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:32.645237 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b433db4-739e-4ee0-a3ed-ecf2ac367301 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:32.743703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:32.752854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b433db4-739e-4ee0-a3ed-ecf2ac367301 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:32.792526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83e643-047a-4b27-9a0f-e32ad1b98808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:32.793386 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83e643-047a-4b27-9a0f-e32ad1b98808 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:32.793672 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83e643-047a-4b27-9a0f-e32ad1b98808 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:32.794042 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83e643-047a-4b27-9a0f-e32ad1b98808 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:32.803272 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83e643-047a-4b27-9a0f-e32ad1b98808 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:32.804952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83e643-047a-4b27-9a0f-e32ad1b98808 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:32.822278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-6d83e643-047a-4b27-9a0f-e32ad1b98808 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:32.822278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:32.822278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:32.822278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:32.822278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:32.822278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:32.822278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:32.822278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:32.822278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:32.822278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:32.822278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:32.822278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:32.822278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:06:32.822278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:06:32.822278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:06:32.822278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:06:32.823651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:32.823651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:32.823651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:06:32.823651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:32.823651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:06:32.823651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:32.823651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:32.823651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:32.823651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:32.823651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:32.823651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:32.823651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:32.823651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:32.823651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:32.823651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:32.823651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:32.823651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:32.823651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:32.825213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:32.825213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:32.825213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:06:32.825213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:06:32.825213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:06:32.825213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:06:32.825213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:06:32.825213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:06:32.825213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:32.825213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:32.825213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 623/1226] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:32.830807 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b44e64-1f07-4dfa-a18a-8aaa05637058 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:32.879475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:32.937687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b44e64-1f07-4dfa-a18a-8aaa05637058 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:32.976899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54a11f-da69-4c9b-ae12-2a8930b4ab53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:32.977585 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54a11f-da69-4c9b-ae12-2a8930b4ab53 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:32.977869 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54a11f-da69-4c9b-ae12-2a8930b4ab53 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:32.978146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54a11f-da69-4c9b-ae12-2a8930b4ab53 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:32.978993 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54a11f-da69-4c9b-ae12-2a8930b4ab53 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:32.979304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54a11f-da69-4c9b-ae12-2a8930b4ab53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:32.980911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54a11f-da69-4c9b-ae12-2a8930b4ab53 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:33.001993 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 604/1227] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Jun 27 21:06:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:33.007812 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83e643-047a-4b27-9a0f-e32ad1b98808 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:33.118960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83e643-047a-4b27-9a0f-e32ad1b98808 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:33.155389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838241db-eb61-42a5-b9ef-e7c7306a231d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:33.156246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838241db-eb61-42a5-b9ef-e7c7306a231d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:33.156526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838241db-eb61-42a5-b9ef-e7c7306a231d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:33.156790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838241db-eb61-42a5-b9ef-e7c7306a231d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:33.171484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838241db-eb61-42a5-b9ef-e7c7306a231d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:33.173029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838241db-eb61-42a5-b9ef-e7c7306a231d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:33.192401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-838241db-eb61-42a5-b9ef-e7c7306a231d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:33.192401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:33.192401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:33.192401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:33.192401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:33.192401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:33.192401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:33.192401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:33.192401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:33.192401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:33.192401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:33.192401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:33.192401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:33.192401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:33.192401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:33.193487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:33.193487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:33.193487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:33.193487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:33.193487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:33.193487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:33.193487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:33.193487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:33.193487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:33.193487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:33.193487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:33.193487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:33.193487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:33.193487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:33.193487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:33.193487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:33.193487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:33.194537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:33.194537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:33.194537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:33.194537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:33.196279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 624/1228] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:33.202942 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54a11f-da69-4c9b-ae12-2a8930b4ab53 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:33.318576 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54a11f-da69-4c9b-ae12-2a8930b4ab53 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:33.321225 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:33.358191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964da735-28ca-46ad-afc5-4809ad5ff702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:33.358990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964da735-28ca-46ad-afc5-4809ad5ff702 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:33.359314 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964da735-28ca-46ad-afc5-4809ad5ff702 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:33.359715 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964da735-28ca-46ad-afc5-4809ad5ff702 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:33.360467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964da735-28ca-46ad-afc5-4809ad5ff702 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:33.360823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964da735-28ca-46ad-afc5-4809ad5ff702 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:33.362514 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964da735-28ca-46ad-afc5-4809ad5ff702 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:33.375331 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 605/1229] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:33.381907 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838241db-eb61-42a5-b9ef-e7c7306a231d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:33.393412 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:33.497497 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838241db-eb61-42a5-b9ef-e7c7306a231d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:33.536956 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557cb339-39ae-48fd-b7af-2a490b8c9d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:33.537872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cb339-39ae-48fd-b7af-2a490b8c9d17 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:33.538152 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cb339-39ae-48fd-b7af-2a490b8c9d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:33.538427 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cb339-39ae-48fd-b7af-2a490b8c9d17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:33.546692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557cb339-39ae-48fd-b7af-2a490b8c9d17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:33.548329 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557cb339-39ae-48fd-b7af-2a490b8c9d17 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:33.563964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-557cb339-39ae-48fd-b7af-2a490b8c9d17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:33.563964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:33.563964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:33.563964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:33.563964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:33.563964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:33.563964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:33.563964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:33.563964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:33.563964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:33.563964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:33.563964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:33.563964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:06:33.563964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:06:33.563964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:06:33.565103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:06:33.565103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:33.565103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:33.565103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:33.565103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:33.565103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:33.565103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:33.565103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:33.565103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:33.565103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:33.565103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:33.565103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:33.565103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:33.565103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:33.565103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:33.565103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:33.565103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:33.566126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:06:33.566126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:06:33.566126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:06:33.566126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:06:33.566126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:06:33.566126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:06:33.566126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:33.566126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:33.566126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 625/1230] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:06:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:33.573673 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964da735-28ca-46ad-afc5-4809ad5ff702 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:33.686624 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964da735-28ca-46ad-afc5-4809ad5ff702 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:33.727034 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f0b66-408a-4768-84c9-acd38b6d10df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:33.727926 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f0b66-408a-4768-84c9-acd38b6d10df None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:33.728265 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f0b66-408a-4768-84c9-acd38b6d10df None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:33.728702 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f0b66-408a-4768-84c9-acd38b6d10df None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:33.729633 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f0b66-408a-4768-84c9-acd38b6d10df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:33.730020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f0b66-408a-4768-84c9-acd38b6d10df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:33.731879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f0b66-408a-4768-84c9-acd38b6d10df None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:33.751705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 606/1231] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:06:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:33.752998 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557cb339-39ae-48fd-b7af-2a490b8c9d17 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:33.823042 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:33.860505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557cb339-39ae-48fd-b7af-2a490b8c9d17 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:33.895288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837fc7c-09d0-4f12-849a-46412f22bcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:33.896228 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837fc7c-09d0-4f12-849a-46412f22bcbc None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:33.896510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837fc7c-09d0-4f12-849a-46412f22bcbc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:33.896784 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837fc7c-09d0-4f12-849a-46412f22bcbc None admin] PATH_INFO: `/v3/projects/dfc29b021d1646819137c89e93f33601/users/1f5fa9b0b0314ee9bb7e16b8464307b1/roles/fb120deba4284112b12de1a251b7da76` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:33.911628 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2837fc7c-09d0-4f12-849a-46412f22bcbc None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc29b021d1646819137c89e93f33601, user_id=1f5fa9b0b0314ee9bb7e16b8464307b1, role_id=fb120deba4284112b12de1a251b7da76)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:33.913105 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2837fc7c-09d0-4f12-849a-46412f22bcbc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:33.927079 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 626/1232] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:06:33 2022] PUT /identity/v3/projects/dfc29b021d1646819137c89e93f33601/users/1f5fa9b0b0314ee9bb7e16b8464307b1/roles/fb120deba4284112b12de1a251b7da76 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:06:36.008454 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858d574-b33c-4a63-9afb-2dc1898d626b None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:36.009029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858d574-b33c-4a63-9afb-2dc1898d626b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:36.009130 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858d574-b33c-4a63-9afb-2dc1898d626b None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:36.010591 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 607/1233] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:06:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:06:36.016435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755864f5-a6fb-4674-89b5-6ede53137585 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:36.016897 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755864f5-a6fb-4674-89b5-6ede53137585 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:36.017003 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755864f5-a6fb-4674-89b5-6ede53137585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:36.034896 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-755864f5-a6fb-4674-89b5-6ede53137585 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:06:36.038258 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-755864f5-a6fb-4674-89b5-6ede53137585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:06:36.110506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 627/1234] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:06:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:06:36.134021 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858d574-b33c-4a63-9afb-2dc1898d626b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:36.162081 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:36.251558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858d574-b33c-4a63-9afb-2dc1898d626b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:36.287984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14b8b2-f18e-4430-94bc-59d8848374a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:36.288799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14b8b2-f18e-4430-94bc-59d8848374a3 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:36.289077 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14b8b2-f18e-4430-94bc-59d8848374a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:36.289339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14b8b2-f18e-4430-94bc-59d8848374a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:36.299331 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14b8b2-f18e-4430-94bc-59d8848374a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:06:36.307183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-bb14b8b2-f18e-4430-94bc-59d8848374a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:06:36.307183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:36.307183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:36.307183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:36.307183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:36.307183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:36.307183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:36.307183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:36.307183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:36.307183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:36.307183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:36.307183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:36.307183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:06:36.307183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:06:36.307183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:06:36.307183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:06:36.308475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:06:36.308475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:06:36.308475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:06:36.308475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:06:36.308475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:36.308475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:36.308475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:36.308475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:36.308475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:36.308475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:36.308475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:36.308475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:36.308475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:36.308475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:36.308475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:36.308475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:36.309539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:36.309539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:36.309539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:36.309539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:36.309539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:06:36.309539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:06:36.309539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:06:36.309539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:06:36.309539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:06:36.309539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:06:36.309539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:06:36.309539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:06:36.313657 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 608/1235] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Jun 27 21:06:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:36.314838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755864f5-a6fb-4674-89b5-6ede53137585 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:36.367562 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:36.426063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755864f5-a6fb-4674-89b5-6ede53137585 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:36.462563 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515ec64-d49e-42ca-bc6d-51b6a4ddd746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:36.463365 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515ec64-d49e-42ca-bc6d-51b6a4ddd746 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:36.463722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515ec64-d49e-42ca-bc6d-51b6a4ddd746 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:36.464093 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515ec64-d49e-42ca-bc6d-51b6a4ddd746 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:36.464880 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515ec64-d49e-42ca-bc6d-51b6a4ddd746 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:36.465235 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515ec64-d49e-42ca-bc6d-51b6a4ddd746 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:36.466920 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515ec64-d49e-42ca-bc6d-51b6a4ddd746 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:36.476205 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 628/1236] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Jun 27 21:06:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:06:36.483118 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14b8b2-f18e-4430-94bc-59d8848374a3 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:36.580639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:36.598137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14b8b2-f18e-4430-94bc-59d8848374a3 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:36.636049 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1537171-9016-4aae-81b8-971abd1ead87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:36.636893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1537171-9016-4aae-81b8-971abd1ead87 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:36.637156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1537171-9016-4aae-81b8-971abd1ead87 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:36.637537 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1537171-9016-4aae-81b8-971abd1ead87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:36.648915 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1537171-9016-4aae-81b8-971abd1ead87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:36.650388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1537171-9016-4aae-81b8-971abd1ead87 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:36.671525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-e1537171-9016-4aae-81b8-971abd1ead87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:36.671525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:36.671525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:36.671525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:36.671525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:36.671525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:36.671525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:36.671525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:36.671525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:36.671525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:36.671525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:36.671525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:36.671525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:36.671525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:36.671525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:36.672708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:36.672708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:36.672708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:36.672708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:36.672708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:36.672708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:36.672708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:36.672708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:36.672708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:36.672708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:36.672708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:36.672708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:36.672708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:36.672708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:36.672708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:36.672708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:36.672708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:36.673788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:36.673788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:36.673788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:36.673788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:06:36.677940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 609/1237] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:36.679245 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515ec64-d49e-42ca-bc6d-51b6a4ddd746 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:36.791639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515ec64-d49e-42ca-bc6d-51b6a4ddd746 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:36.801493 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:36.827306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045031f8-11d1-4b39-b6c8-ac7b255f898c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:36.828014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045031f8-11d1-4b39-b6c8-ac7b255f898c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:36.828293 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045031f8-11d1-4b39-b6c8-ac7b255f898c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:36.828552 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045031f8-11d1-4b39-b6c8-ac7b255f898c None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:36.829279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045031f8-11d1-4b39-b6c8-ac7b255f898c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:36.829596 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045031f8-11d1-4b39-b6c8-ac7b255f898c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:36.831053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045031f8-11d1-4b39-b6c8-ac7b255f898c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:36.842696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 629/1238] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:36.848548 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1537171-9016-4aae-81b8-971abd1ead87 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:36.954830 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1537171-9016-4aae-81b8-971abd1ead87 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:36.996545 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b15c1b-f089-47a9-ac80-b36b763af1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:36.997380 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b15c1b-f089-47a9-ac80-b36b763af1cf None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:36.997663 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b15c1b-f089-47a9-ac80-b36b763af1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:36.997936 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b15c1b-f089-47a9-ac80-b36b763af1cf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:37.000986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:37.007709 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b15c1b-f089-47a9-ac80-b36b763af1cf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:37.009410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b15c1b-f089-47a9-ac80-b36b763af1cf None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:37.024158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-e7b15c1b-f089-47a9-ac80-b36b763af1cf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:37.024158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:37.024158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:37.024158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:37.024158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:37.024158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:37.024158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:37.024158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:37.024158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:37.024158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:37.024158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:37.024158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:37.024158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:06:37.024158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:06:37.024158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:06:37.024158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:06:37.025217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:37.025217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:37.025217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:06:37.025217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:37.025217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:06:37.025217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:37.025217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:37.025217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:37.025217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:37.025217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:37.025217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:37.025217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:37.025217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:37.025217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:37.025217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:37.025217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:37.025217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:37.025217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:37.026287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:37.026287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:37.026287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:06:37.026287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:06:37.026287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:06:37.026287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:06:37.026287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:06:37.026287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:06:37.026287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:37.026287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:06:37.031345 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 610/1239] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:37.032519 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045031f8-11d1-4b39-b6c8-ac7b255f898c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:37.141630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045031f8-11d1-4b39-b6c8-ac7b255f898c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:37.177925 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c435a-73c3-4d97-96e6-0d1f93ef0133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:37.178602 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c435a-73c3-4d97-96e6-0d1f93ef0133 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:37.178894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c435a-73c3-4d97-96e6-0d1f93ef0133 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:37.179165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c435a-73c3-4d97-96e6-0d1f93ef0133 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:37.180026 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c435a-73c3-4d97-96e6-0d1f93ef0133 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:37.180332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c435a-73c3-4d97-96e6-0d1f93ef0133 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:37.181778 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c435a-73c3-4d97-96e6-0d1f93ef0133 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:37.204626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 630/1240] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Jun 27 21:06:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:37.210748 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b15c1b-f089-47a9-ac80-b36b763af1cf None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:37.322353 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b15c1b-f089-47a9-ac80-b36b763af1cf None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:37.356755 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82296a-963d-4604-93fc-1ac555ccca3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:37.357439 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82296a-963d-4604-93fc-1ac555ccca3a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:37.357705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82296a-963d-4604-93fc-1ac555ccca3a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:37.357978 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82296a-963d-4604-93fc-1ac555ccca3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:37.433091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:37.440702 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82296a-963d-4604-93fc-1ac555ccca3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:37.442169 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82296a-963d-4604-93fc-1ac555ccca3a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:37.553183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 611/1241] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:06:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:06:37.559354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c435a-73c3-4d97-96e6-0d1f93ef0133 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:37.593159 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:37.669843 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c435a-73c3-4d97-96e6-0d1f93ef0133 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:37.705372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3656f20-829c-413a-838f-e90469bbd863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:37.706114 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3656f20-829c-413a-838f-e90469bbd863 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:37.706434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3656f20-829c-413a-838f-e90469bbd863 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:37.706708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3656f20-829c-413a-838f-e90469bbd863 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:37.718042 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3656f20-829c-413a-838f-e90469bbd863 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:37.719513 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3656f20-829c-413a-838f-e90469bbd863 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:37.741384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-d3656f20-829c-413a-838f-e90469bbd863 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:37.741384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:37.741384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:37.741384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:37.741384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:37.741384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:37.741384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:37.741384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:37.741384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:37.741384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:37.741384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:37.741384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:37.741384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:37.741384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:37.741384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:37.742434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:37.742434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:37.742434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:37.742434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:37.742434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:37.742434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:37.742434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:37.742434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:37.742434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:37.742434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:37.742434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:37.742434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:37.742434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:37.742434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:37.742434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:37.742434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:37.742434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:37.743500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:37.743500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:37.743500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:37.743500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:37.743500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 631/1242] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:37.748864 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82296a-963d-4604-93fc-1ac555ccca3a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:37.851072 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:37.857933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82296a-963d-4604-93fc-1ac555ccca3a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:37.895822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d87daa-6613-473f-bcbb-bab391f30400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:37.896482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d87daa-6613-473f-bcbb-bab391f30400 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:37.896768 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d87daa-6613-473f-bcbb-bab391f30400 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:37.897039 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d87daa-6613-473f-bcbb-bab391f30400 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:37.897764 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d87daa-6613-473f-bcbb-bab391f30400 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:37.898064 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d87daa-6613-473f-bcbb-bab391f30400 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:37.899446 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d87daa-6613-473f-bcbb-bab391f30400 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:37.912163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 612/1243] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:06:37.917978 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3656f20-829c-413a-838f-e90469bbd863 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:38.005524 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:38.032123 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3656f20-829c-413a-838f-e90469bbd863 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:38.068610 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23f1b4-f1a2-49c0-adc4-c70b6017e51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:38.070132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23f1b4-f1a2-49c0-adc4-c70b6017e51c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:38.070411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23f1b4-f1a2-49c0-adc4-c70b6017e51c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:38.070679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23f1b4-f1a2-49c0-adc4-c70b6017e51c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:38.078028 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23f1b4-f1a2-49c0-adc4-c70b6017e51c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:38.079757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23f1b4-f1a2-49c0-adc4-c70b6017e51c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:38.094846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-6f23f1b4-f1a2-49c0-adc4-c70b6017e51c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:38.094846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:38.094846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:38.094846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:38.094846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:38.094846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:38.094846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:38.094846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:38.094846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:38.094846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:38.094846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:38.094846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:38.094846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:06:38.094846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:06:38.094846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:06:38.095823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:06:38.095823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:38.095823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:38.095823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:38.095823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:38.095823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:38.095823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:38.095823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:38.095823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:38.095823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:38.095823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:38.095823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:38.095823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:38.095823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:38.095823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:38.095823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:38.095823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:38.096691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:06:38.096691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:06:38.096691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:06:38.096691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:06:38.096691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:06:38.096691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:06:38.096691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:38.096691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:38.096691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 632/1244] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:06:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:38.102483 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d87daa-6613-473f-bcbb-bab391f30400 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:38.214462 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d87daa-6613-473f-bcbb-bab391f30400 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:38.249964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1efba-f905-4a3b-a694-191ff5654bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:38.250588 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1efba-f905-4a3b-a694-191ff5654bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:38.250982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1efba-f905-4a3b-a694-191ff5654bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:38.251329 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1efba-f905-4a3b-a694-191ff5654bd8 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:38.252226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1efba-f905-4a3b-a694-191ff5654bd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:38.252582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1efba-f905-4a3b-a694-191ff5654bd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:38.254289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1efba-f905-4a3b-a694-191ff5654bd8 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:38.267866 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 613/1245] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:06:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:38.274603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23f1b4-f1a2-49c0-adc4-c70b6017e51c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:38.381996 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23f1b4-f1a2-49c0-adc4-c70b6017e51c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:38.417262 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2acc3e5-d1db-47c1-b4c3-1ba2c8249de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:38.418024 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acc3e5-d1db-47c1-b4c3-1ba2c8249de7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:38.418328 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acc3e5-d1db-47c1-b4c3-1ba2c8249de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:38.418626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acc3e5-d1db-47c1-b4c3-1ba2c8249de7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:38.419510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2acc3e5-d1db-47c1-b4c3-1ba2c8249de7 None admin] RBAC: Adding query filter params (role.id=fb120deba4284112b12de1a251b7da76, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=1f5fa9b0b0314ee9bb7e16b8464307b1) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:38.419938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2acc3e5-d1db-47c1-b4c3-1ba2c8249de7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:38.421608 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2acc3e5-d1db-47c1-b4c3-1ba2c8249de7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:38.427903 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 633/1246] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:06:38 2022] GET /identity/v3/role_assignments?user.id=1f5fa9b0b0314ee9bb7e16b8464307b1&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=fb120deba4284112b12de1a251b7da76 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:40.696023 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0f039-32ad-460f-8c8c-c22ea1c20322 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:40.696583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0f039-32ad-460f-8c8c-c22ea1c20322 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:40.696583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0f039-32ad-460f-8c8c-c22ea1c20322 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:40.698200 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 614/1247] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:06:40.704162 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680ea58-0425-4064-b246-1ad29131af48 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:40.704750 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680ea58-0425-4064-b246-1ad29131af48 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:40.704750 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680ea58-0425-4064-b246-1ad29131af48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:40.710423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:40.725731 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5680ea58-0425-4064-b246-1ad29131af48 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:06:40.727254 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5680ea58-0425-4064-b246-1ad29131af48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:06:40.804509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 634/1248] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:06:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:06:40.830037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb0f039-32ad-460f-8c8c-c22ea1c20322 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:40.849689 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:40.940694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb0f039-32ad-460f-8c8c-c22ea1c20322 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:40.979446 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19635649-ffbb-4101-a278-f7c47661f86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:40.980243 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19635649-ffbb-4101-a278-f7c47661f86f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:40.980527 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19635649-ffbb-4101-a278-f7c47661f86f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:40.980832 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19635649-ffbb-4101-a278-f7c47661f86f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:40.992186 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19635649-ffbb-4101-a278-f7c47661f86f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:06:41.000083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-19635649-ffbb-4101-a278-f7c47661f86f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:06:41.000083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:41.000083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:41.000083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:41.000083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:41.000083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:41.000083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:41.000083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:41.000083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:41.000083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:41.000083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:41.000083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:41.000083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:06:41.000083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:06:41.000083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:06:41.000083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:06:41.001285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:06:41.001285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:06:41.001285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:06:41.001285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:06:41.001285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:41.001285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:41.001285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:41.001285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:41.001285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:41.001285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:41.001285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:41.001285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:41.001285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:41.001285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:41.001285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:41.001285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:41.002146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:41.002146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:41.002146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:41.002146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:41.002146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:06:41.002146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:06:41.002146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:06:41.002146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:06:41.002146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:06:41.002146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:06:41.002146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:06:41.002146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:06:41.002146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 615/1249] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:06:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:41.007635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680ea58-0425-4064-b246-1ad29131af48 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:41.114168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680ea58-0425-4064-b246-1ad29131af48 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:41.147356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:41.153878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf326f-a4e8-4618-be3b-37cb7e6f73fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:41.154637 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf326f-a4e8-4618-be3b-37cb7e6f73fa None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:41.154919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf326f-a4e8-4618-be3b-37cb7e6f73fa None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:41.155256 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf326f-a4e8-4618-be3b-37cb7e6f73fa None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:41.156215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf326f-a4e8-4618-be3b-37cb7e6f73fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:41.156578 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf326f-a4e8-4618-be3b-37cb7e6f73fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:41.158204 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf326f-a4e8-4618-be3b-37cb7e6f73fa None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:41.167209 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 635/1250] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:06:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:41.173323 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19635649-ffbb-4101-a278-f7c47661f86f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:41.256387 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:41.281616 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19635649-ffbb-4101-a278-f7c47661f86f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:41.319498 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12b8d1-1b75-4bcc-bdce-14abd1b5d094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:41.320431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12b8d1-1b75-4bcc-bdce-14abd1b5d094 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:41.320681 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12b8d1-1b75-4bcc-bdce-14abd1b5d094 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:41.320915 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12b8d1-1b75-4bcc-bdce-14abd1b5d094 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:41.332336 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12b8d1-1b75-4bcc-bdce-14abd1b5d094 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:41.333794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12b8d1-1b75-4bcc-bdce-14abd1b5d094 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:41.352907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-4a12b8d1-1b75-4bcc-bdce-14abd1b5d094 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:41.352907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:41.352907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:41.352907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:41.352907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:41.352907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:41.352907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:41.352907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:41.352907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:41.352907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:41.352907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:41.352907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:41.352907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:41.352907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:41.352907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:41.354091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:41.354091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:41.354091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:41.354091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:41.354091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:41.354091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:41.354091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:41.354091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:41.354091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:41.354091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:41.354091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:41.354091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:41.354091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:41.354091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:41.354091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:41.354091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:41.354091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:41.355185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:41.355185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:41.355185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:41.355185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:06:41.355185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 616/1251] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:06:41.360053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf326f-a4e8-4618-be3b-37cb7e6f73fa None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:41.478238 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf326f-a4e8-4618-be3b-37cb7e6f73fa None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:41.514099 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c059a13-6bfb-401f-877b-04b3dfcfbb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:41.514700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c059a13-6bfb-401f-877b-04b3dfcfbb3b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:41.514970 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c059a13-6bfb-401f-877b-04b3dfcfbb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:41.515239 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c059a13-6bfb-401f-877b-04b3dfcfbb3b None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:41.515994 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c059a13-6bfb-401f-877b-04b3dfcfbb3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:41.516301 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c059a13-6bfb-401f-877b-04b3dfcfbb3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:41.517692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c059a13-6bfb-401f-877b-04b3dfcfbb3b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:41.530792 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 636/1252] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:06:41.536771 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12b8d1-1b75-4bcc-bdce-14abd1b5d094 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:41.655237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12b8d1-1b75-4bcc-bdce-14abd1b5d094 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:41.683365 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:41.692482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c38a4-c287-43f9-89ff-01ab0ebbd468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:41.693374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c38a4-c287-43f9-89ff-01ab0ebbd468 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:41.693565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c38a4-c287-43f9-89ff-01ab0ebbd468 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:41.693836 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c38a4-c287-43f9-89ff-01ab0ebbd468 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:41.701960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c38a4-c287-43f9-89ff-01ab0ebbd468 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:41.703371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c38a4-c287-43f9-89ff-01ab0ebbd468 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:41.718328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-036c38a4-c287-43f9-89ff-01ab0ebbd468 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:41.718328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:41.718328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:41.718328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:41.718328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:41.718328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:41.718328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:41.718328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:41.718328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:41.718328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:41.718328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:41.718328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:41.718328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:06:41.718328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:06:41.718328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:06:41.718328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:06:41.720100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:41.720100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:41.720100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:06:41.720100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:41.720100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:06:41.720100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:41.720100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:41.720100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:41.720100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:41.720100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:41.720100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:41.720100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:41.720100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:41.720100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:41.720100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:41.720100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:41.720100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:41.720100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:41.721080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:41.721080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:41.721080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:06:41.721080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:06:41.721080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:06:41.721080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:06:41.721080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:06:41.721080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:06:41.721080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:41.721080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:06:41.721080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 617/1253] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:41.725823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c059a13-6bfb-401f-877b-04b3dfcfbb3b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:41.793636 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:41.840915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c059a13-6bfb-401f-877b-04b3dfcfbb3b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:41.878302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638da00b-9e5e-4ae7-9843-222c1c7eddfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:41.879119 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638da00b-9e5e-4ae7-9843-222c1c7eddfb None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:41.879434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638da00b-9e5e-4ae7-9843-222c1c7eddfb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:41.879761 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638da00b-9e5e-4ae7-9843-222c1c7eddfb None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:41.880701 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638da00b-9e5e-4ae7-9843-222c1c7eddfb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:41.881075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638da00b-9e5e-4ae7-9843-222c1c7eddfb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:41.882644 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638da00b-9e5e-4ae7-9843-222c1c7eddfb None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:41.900421 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 637/1254] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Jun 27 21:06:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:41.906814 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c38a4-c287-43f9-89ff-01ab0ebbd468 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:42.015467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c38a4-c287-43f9-89ff-01ab0ebbd468 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:42.053649 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f83ce-75a6-413f-8556-845456d4fd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:42.054247 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f83ce-75a6-413f-8556-845456d4fd7b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:42.054523 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f83ce-75a6-413f-8556-845456d4fd7b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:42.054803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f83ce-75a6-413f-8556-845456d4fd7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:42.116755 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:42.132249 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f83ce-75a6-413f-8556-845456d4fd7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:42.133878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f83ce-75a6-413f-8556-845456d4fd7b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:42.237929 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 618/1255] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:06:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:06:42.246386 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638da00b-9e5e-4ae7-9843-222c1c7eddfb None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:42.362548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638da00b-9e5e-4ae7-9843-222c1c7eddfb None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:42.377575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:42.401438 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9943e-5b44-4376-bf41-71d92996f561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:42.402414 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9943e-5b44-4376-bf41-71d92996f561 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:42.402731 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9943e-5b44-4376-bf41-71d92996f561 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:42.403010 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9943e-5b44-4376-bf41-71d92996f561 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:42.415190 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9943e-5b44-4376-bf41-71d92996f561 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:42.417041 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9943e-5b44-4376-bf41-71d92996f561 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:42.438128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-e0a9943e-5b44-4376-bf41-71d92996f561 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:42.438128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:42.438128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:42.438128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:42.438128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:42.438128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:42.438128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:42.438128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:42.438128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:42.438128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:42.438128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:42.438128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:42.438128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:42.438128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:42.438128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:42.439378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:42.439378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:42.439378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:42.439378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:42.439378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:42.439378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:42.439378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:42.439378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:42.439378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:42.439378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:42.439378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:42.439378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:42.439378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:42.439378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:42.439378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:42.439378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:42.439378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:42.440483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:42.440483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:42.440483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:42.440483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:42.440483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 638/1256] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:42.446606 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f83ce-75a6-413f-8556-845456d4fd7b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:42.550082 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:42.564259 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f83ce-75a6-413f-8556-845456d4fd7b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:42.600074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddc451-e67b-444d-86e8-60cf7185c26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:42.600829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddc451-e67b-444d-86e8-60cf7185c26a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:42.601217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddc451-e67b-444d-86e8-60cf7185c26a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:42.601570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddc451-e67b-444d-86e8-60cf7185c26a None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:42.602366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ddc451-e67b-444d-86e8-60cf7185c26a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:42.602705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ddc451-e67b-444d-86e8-60cf7185c26a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:42.604171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ddc451-e67b-444d-86e8-60cf7185c26a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:42.616426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 619/1257] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:42.622409 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9943e-5b44-4376-bf41-71d92996f561 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:42.726457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9943e-5b44-4376-bf41-71d92996f561 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:42.766704 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb7425-120d-4315-bc2b-b8347750f3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:42.767578 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb7425-120d-4315-bc2b-b8347750f3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:42.767919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb7425-120d-4315-bc2b-b8347750f3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:42.768203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb7425-120d-4315-bc2b-b8347750f3a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:42.776669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb7425-120d-4315-bc2b-b8347750f3a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:42.778732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb7425-120d-4315-bc2b-b8347750f3a2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:42.796221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-dfcb7425-120d-4315-bc2b-b8347750f3a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:42.796221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:42.796221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:42.796221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:42.796221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:42.796221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:42.796221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:42.796221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:42.796221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:42.796221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:42.796221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:42.796221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:42.796221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:06:42.796221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:06:42.796221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:06:42.797241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:06:42.797241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:42.797241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:42.797241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:42.797241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:42.797241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:42.797241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:42.797241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:42.797241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:42.797241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:42.797241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:42.797241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:42.797241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:42.797241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:42.797241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:42.797241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:42.797241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:42.798318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:06:42.798318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:06:42.798318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:06:42.798318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:06:42.798318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:06:42.798318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:06:42.798318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:42.798318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:42.798318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 639/1258] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:06:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:42.804778 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddc451-e67b-444d-86e8-60cf7185c26a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:42.912773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddc451-e67b-444d-86e8-60cf7185c26a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:42.946581 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d091db-d97d-4bb7-aaf2-091ccda87611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:42.947285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d091db-d97d-4bb7-aaf2-091ccda87611 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:42.947630 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d091db-d97d-4bb7-aaf2-091ccda87611 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:42.947907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d091db-d97d-4bb7-aaf2-091ccda87611 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:42.948688 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d091db-d97d-4bb7-aaf2-091ccda87611 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:42.948995 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d091db-d97d-4bb7-aaf2-091ccda87611 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:42.950432 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d091db-d97d-4bb7-aaf2-091ccda87611 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:42.962959 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 620/1259] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:06:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:06:42.969441 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb7425-120d-4315-bc2b-b8347750f3a2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:43.000091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:43.080203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb7425-120d-4315-bc2b-b8347750f3a2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:43.116700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0964240-399a-4522-ab23-2ad4af3d72a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:43.117446 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0964240-399a-4522-ab23-2ad4af3d72a1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:43.117767 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0964240-399a-4522-ab23-2ad4af3d72a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:43.118096 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0964240-399a-4522-ab23-2ad4af3d72a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:43.119061 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0964240-399a-4522-ab23-2ad4af3d72a1 None admin] RBAC: Adding query filter params (role.id=5c3c095bb53943eca95c4e049cd8cbf5, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=1f5fa9b0b0314ee9bb7e16b8464307b1) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:43.119404 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0964240-399a-4522-ab23-2ad4af3d72a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:43.121178 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0964240-399a-4522-ab23-2ad4af3d72a1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:43.127308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 640/1260] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:06:42 2022] GET /identity/v3/role_assignments?user.id=1f5fa9b0b0314ee9bb7e16b8464307b1&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=5c3c095bb53943eca95c4e049cd8cbf5 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:06:45.112868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e55ba0-3a85-44be-8432-1c29abefe45a None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:45.113457 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e55ba0-3a85-44be-8432-1c29abefe45a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:45.113457 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e55ba0-3a85-44be-8432-1c29abefe45a None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:45.114798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 621/1261] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:06:45.121378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82a813-9445-4239-b1c8-e1118936d6ea None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:45.121807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82a813-9445-4239-b1c8-e1118936d6ea None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:45.121948 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82a813-9445-4239-b1c8-e1118936d6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:45.150990 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-fc82a813-9445-4239-b1c8-e1118936d6ea None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:06:45.152248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fc82a813-9445-4239-b1c8-e1118936d6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:06:45.228419 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 641/1262] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:06:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:06:45.252099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e55ba0-3a85-44be-8432-1c29abefe45a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:45.263524 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:45.359362 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e55ba0-3a85-44be-8432-1c29abefe45a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:45.395704 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2dc7e-62ca-421e-a03f-d7001c2e8706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:45.396501 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2dc7e-62ca-421e-a03f-d7001c2e8706 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:45.396776 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2dc7e-62ca-421e-a03f-d7001c2e8706 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:45.397042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2dc7e-62ca-421e-a03f-d7001c2e8706 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:45.408466 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2dc7e-62ca-421e-a03f-d7001c2e8706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:06:45.416119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-3cb2dc7e-62ca-421e-a03f-d7001c2e8706 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:06:45.416119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:45.416119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:45.416119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:45.416119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:45.416119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:45.416119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:45.416119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:45.416119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:45.416119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:45.416119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:45.416119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:45.416119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:06:45.416119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:06:45.416119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:06:45.416119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:06:45.417191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:06:45.417191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:06:45.417191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:06:45.417191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:06:45.417191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:45.417191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:45.417191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:45.417191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:45.417191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:45.417191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:45.417191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:45.417191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:45.417191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:45.417191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:45.417191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:45.417191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:45.418158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:45.418158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:45.418158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:45.418158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:45.418158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:06:45.418158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:06:45.418158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:06:45.418158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:06:45.418158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:06:45.418158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:06:45.418158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:06:45.418158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:06:45.418158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 622/1263] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:06:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:45.424544 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82a813-9445-4239-b1c8-e1118936d6ea None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:45.447170 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:45.542831 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82a813-9445-4239-b1c8-e1118936d6ea None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:45.578659 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f76b4d-9b51-4203-a360-0da2c9eb0154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:45.579329 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f76b4d-9b51-4203-a360-0da2c9eb0154 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:45.579650 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f76b4d-9b51-4203-a360-0da2c9eb0154 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:45.579922 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f76b4d-9b51-4203-a360-0da2c9eb0154 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:45.580677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f76b4d-9b51-4203-a360-0da2c9eb0154 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:45.580987 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f76b4d-9b51-4203-a360-0da2c9eb0154 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:45.582387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f76b4d-9b51-4203-a360-0da2c9eb0154 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:45.590621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 642/1264] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:06:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:45.596733 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2dc7e-62ca-421e-a03f-d7001c2e8706 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:45.666014 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:45.700722 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2dc7e-62ca-421e-a03f-d7001c2e8706 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:45.738287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f59d2-2f6d-40a7-850e-ddb3cbe80956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:45.739091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f59d2-2f6d-40a7-850e-ddb3cbe80956 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:45.739365 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f59d2-2f6d-40a7-850e-ddb3cbe80956 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:45.739669 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f59d2-2f6d-40a7-850e-ddb3cbe80956 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:45.751173 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f59d2-2f6d-40a7-850e-ddb3cbe80956 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:45.752672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f59d2-2f6d-40a7-850e-ddb3cbe80956 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:45.772985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-902f59d2-2f6d-40a7-850e-ddb3cbe80956 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:45.772985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:45.772985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:45.772985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:45.772985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:45.772985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:45.772985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:45.772985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:45.772985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:45.772985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:45.772985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:45.772985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:45.772985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:45.772985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:45.772985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:45.774051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:45.774051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:45.774051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:45.774051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:45.774051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:45.774051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:45.774051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:45.774051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:45.774051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:45.774051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:45.774051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:45.774051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:45.774051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:45.774051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:45.774051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:45.774051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:45.774051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:45.775073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:45.775073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:45.775073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:45.775073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:06:45.775073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 623/1265] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:45.780719 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f76b4d-9b51-4203-a360-0da2c9eb0154 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:45.869471 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:45.892256 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f76b4d-9b51-4203-a360-0da2c9eb0154 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:45.928451 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd233cc1-8a7c-4f85-800f-7c4cd778b552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:45.929107 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd233cc1-8a7c-4f85-800f-7c4cd778b552 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:45.929388 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd233cc1-8a7c-4f85-800f-7c4cd778b552 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:45.929657 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd233cc1-8a7c-4f85-800f-7c4cd778b552 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:45.930368 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd233cc1-8a7c-4f85-800f-7c4cd778b552 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:45.930671 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd233cc1-8a7c-4f85-800f-7c4cd778b552 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:45.932152 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd233cc1-8a7c-4f85-800f-7c4cd778b552 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:45.943851 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 643/1266] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:45.949870 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f59d2-2f6d-40a7-850e-ddb3cbe80956 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:46.058783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f59d2-2f6d-40a7-850e-ddb3cbe80956 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:46.078472 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:46.095091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b2168-ac40-45e6-828e-05d1dd0d5533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:46.095934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b2168-ac40-45e6-828e-05d1dd0d5533 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:46.096218 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b2168-ac40-45e6-828e-05d1dd0d5533 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:46.096486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b2168-ac40-45e6-828e-05d1dd0d5533 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:46.105176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b2168-ac40-45e6-828e-05d1dd0d5533 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:46.106636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b2168-ac40-45e6-828e-05d1dd0d5533 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:46.122092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-a79b2168-ac40-45e6-828e-05d1dd0d5533 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:46.122092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:46.122092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:46.122092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:46.122092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:46.122092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:46.122092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:46.122092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:46.122092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:46.122092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:46.122092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:46.122092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:46.122092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:06:46.122092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:06:46.122092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:06:46.122092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:06:46.127212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:46.127212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:46.127212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:06:46.127212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:46.127212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:06:46.127212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:46.127212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:46.127212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:46.127212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:46.127212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:46.127212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:46.127212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:46.127212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:46.127212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:46.127212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:46.127212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:46.127212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:46.127212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:46.128957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:46.128957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:46.128957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:06:46.128957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:06:46.128957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:06:46.128957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:06:46.128957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:06:46.128957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:06:46.128957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:46.128957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:06:46.128957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 624/1267] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:46.132209 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd233cc1-8a7c-4f85-800f-7c4cd778b552 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:46.243578 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd233cc1-8a7c-4f85-800f-7c4cd778b552 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:46.279057 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39864f9f-e3af-4e7e-819d-1cb6a0c4f2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:46.279742 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39864f9f-e3af-4e7e-819d-1cb6a0c4f2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:46.280022 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39864f9f-e3af-4e7e-819d-1cb6a0c4f2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:46.280290 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39864f9f-e3af-4e7e-819d-1cb6a0c4f2c1 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:46.281124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39864f9f-e3af-4e7e-819d-1cb6a0c4f2c1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:46.281428 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39864f9f-e3af-4e7e-819d-1cb6a0c4f2c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:46.282884 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39864f9f-e3af-4e7e-819d-1cb6a0c4f2c1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:46.291662 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:46.300464 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 644/1268] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Jun 27 21:06:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:46.307141 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b2168-ac40-45e6-828e-05d1dd0d5533 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:46.425351 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b2168-ac40-45e6-828e-05d1dd0d5533 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:46.461431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b5e88-e31f-4ce4-bad4-e361d57c2644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:46.462221 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b5e88-e31f-4ce4-bad4-e361d57c2644 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:46.462502 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b5e88-e31f-4ce4-bad4-e361d57c2644 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:46.462758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b5e88-e31f-4ce4-bad4-e361d57c2644 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:46.474555 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b5e88-e31f-4ce4-bad4-e361d57c2644 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:46.476057 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b5e88-e31f-4ce4-bad4-e361d57c2644 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:46.493477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-d05b5e88-e31f-4ce4-bad4-e361d57c2644 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:46.493477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:46.493477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:46.493477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:46.493477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:46.493477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:46.493477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:46.493477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:46.493477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:46.493477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:46.493477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:46.493477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:46.493477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:46.493477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:46.493477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:46.494570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:46.494570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:46.494570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:46.494570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:46.494570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:46.494570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:46.494570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:46.494570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:46.494570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:46.494570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:46.494570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:46.494570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:46.494570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:46.494570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:46.494570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:46.494570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:46.494570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:46.495477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:46.495477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:46.495477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:46.495477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:06:46.495477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 625/1269] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:46.503410 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39864f9f-e3af-4e7e-819d-1cb6a0c4f2c1 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:46.607901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39864f9f-e3af-4e7e-819d-1cb6a0c4f2c1 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:46.642332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb016e-f63d-4743-9625-e781c79fac7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:46.642966 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb016e-f63d-4743-9625-e781c79fac7b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:46.643244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb016e-f63d-4743-9625-e781c79fac7b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:46.643513 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb016e-f63d-4743-9625-e781c79fac7b None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:46.644263 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb016e-f63d-4743-9625-e781c79fac7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:46.644567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb016e-f63d-4743-9625-e781c79fac7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:46.645943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb016e-f63d-4743-9625-e781c79fac7b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:46.657307 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 645/1270] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:46.666161 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b5e88-e31f-4ce4-bad4-e361d57c2644 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:46.709511 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:46.792795 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b5e88-e31f-4ce4-bad4-e361d57c2644 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:46.827847 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf602dae-e515-4d38-abab-f36610fa963e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:46.828645 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf602dae-e515-4d38-abab-f36610fa963e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:46.828926 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf602dae-e515-4d38-abab-f36610fa963e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:46.829203 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf602dae-e515-4d38-abab-f36610fa963e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:46.836295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf602dae-e515-4d38-abab-f36610fa963e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:46.837751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf602dae-e515-4d38-abab-f36610fa963e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:46.852183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-cf602dae-e515-4d38-abab-f36610fa963e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:46.852183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:46.852183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:46.852183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:46.852183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:46.852183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:46.852183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:46.852183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:46.852183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:46.852183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:46.852183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:46.852183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:46.852183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:06:46.852183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:06:46.852183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:06:46.853167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:06:46.853167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:46.853167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:46.853167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:46.853167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:46.853167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:46.853167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:46.853167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:46.853167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:46.853167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:46.853167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:46.853167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:46.853167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:46.853167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:46.853167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:46.853167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:46.853167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:46.854435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:06:46.854435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:06:46.854435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:06:46.854435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:06:46.854435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:06:46.854435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:06:46.854435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:46.854435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:06:46.854435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 626/1271] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:06:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:46.860133 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb016e-f63d-4743-9625-e781c79fac7b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:46.926679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:46.973511 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb016e-f63d-4743-9625-e781c79fac7b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:47.008705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e6bdf-4cf8-4dc4-a88a-5594cf17e060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:47.009373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e6bdf-4cf8-4dc4-a88a-5594cf17e060 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:47.009656 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e6bdf-4cf8-4dc4-a88a-5594cf17e060 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:47.009983 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e6bdf-4cf8-4dc4-a88a-5594cf17e060 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:47.010775 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e6bdf-4cf8-4dc4-a88a-5594cf17e060 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:47.011087 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e6bdf-4cf8-4dc4-a88a-5594cf17e060 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:47.012587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e6bdf-4cf8-4dc4-a88a-5594cf17e060 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:47.024599 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 646/1272] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:06:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:47.034533 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf602dae-e515-4d38-abab-f36610fa963e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:47.136199 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:47.146728 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf602dae-e515-4d38-abab-f36610fa963e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:47.182782 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbcfa3-be3d-407c-8e68-06a642111af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:47.183695 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbcfa3-be3d-407c-8e68-06a642111af0 None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:47.183973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbcfa3-be3d-407c-8e68-06a642111af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:47.184242 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbcfa3-be3d-407c-8e68-06a642111af0 None admin] PATH_INFO: `/v3/projects/dfc29b021d1646819137c89e93f33601/users/1f5fa9b0b0314ee9bb7e16b8464307b1/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:47.186296 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cbcfa3-be3d-407c-8e68-06a642111af0 None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc29b021d1646819137c89e93f33601, user_id=1f5fa9b0b0314ee9bb7e16b8464307b1, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:47.187774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cbcfa3-be3d-407c-8e68-06a642111af0 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:47.202661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 627/1273] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Jun 27 21:06:47 2022] PUT /identity/v3/projects/dfc29b021d1646819137c89e93f33601/users/1f5fa9b0b0314ee9bb7e16b8464307b1/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:06:49.192728 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60981eeb-c521-44e3-ab85-fb6781e5730b None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:49.193566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60981eeb-c521-44e3-ab85-fb6781e5730b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:49.193566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60981eeb-c521-44e3-ab85-fb6781e5730b None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:49.195852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 647/1274] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:06:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:06:49.203766 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15b7d1-c4a2-41eb-9dc7-06b4a1a5bf99 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:49.204235 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15b7d1-c4a2-41eb-9dc7-06b4a1a5bf99 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:49.204326 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15b7d1-c4a2-41eb-9dc7-06b4a1a5bf99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:49.222638 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-fc15b7d1-c4a2-41eb-9dc7-06b4a1a5bf99 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:06:49.223848 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fc15b7d1-c4a2-41eb-9dc7-06b4a1a5bf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:06:49.295724 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 628/1275] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:06:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:06:49.320039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60981eeb-c521-44e3-ab85-fb6781e5730b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:49.428537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60981eeb-c521-44e3-ab85-fb6781e5730b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:49.446185 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:49.467444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1377c-2f2c-4598-9173-c164260d0902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:49.468287 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1377c-2f2c-4598-9173-c164260d0902 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:49.468570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1377c-2f2c-4598-9173-c164260d0902 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:49.468840 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1377c-2f2c-4598-9173-c164260d0902 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:49.479955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1377c-2f2c-4598-9173-c164260d0902 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:06:49.487639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-c6e1377c-2f2c-4598-9173-c164260d0902 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:06:49.487639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:49.487639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:49.487639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:49.487639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:49.487639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:49.487639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:49.487639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:49.487639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:49.487639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:49.487639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:49.487639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:49.487639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:06:49.487639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:06:49.487639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:06:49.487639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:06:49.488709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:06:49.488709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:06:49.488709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:06:49.488709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:06:49.488709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:49.488709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:49.488709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:49.488709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:49.488709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:49.488709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:49.488709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:49.488709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:49.488709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:49.488709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:49.488709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:49.488709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:49.489630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:49.489630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:49.489630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:49.489630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:49.489630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:06:49.489630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:06:49.489630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:06:49.489630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:06:49.489630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:06:49.489630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:06:49.489630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:06:49.489630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:49.489630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 648/1276] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Jun 27 21:06:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:49.495123 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15b7d1-c4a2-41eb-9dc7-06b4a1a5bf99 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:49.587298 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:49.608336 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15b7d1-c4a2-41eb-9dc7-06b4a1a5bf99 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:49.645422 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22107327-61e8-4a80-8f47-f1466e337079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:49.646088 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22107327-61e8-4a80-8f47-f1466e337079 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:49.646363 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22107327-61e8-4a80-8f47-f1466e337079 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:49.646635 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22107327-61e8-4a80-8f47-f1466e337079 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:49.647377 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22107327-61e8-4a80-8f47-f1466e337079 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:49.647709 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22107327-61e8-4a80-8f47-f1466e337079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:49.649085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22107327-61e8-4a80-8f47-f1466e337079 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:49.657617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 629/1277] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Jun 27 21:06:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:49.664163 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1377c-2f2c-4598-9173-c164260d0902 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:49.796227 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1377c-2f2c-4598-9173-c164260d0902 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:49.836537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be591a1-2a6b-4782-bef8-892e74a8040b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:49.837397 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be591a1-2a6b-4782-bef8-892e74a8040b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:49.837620 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be591a1-2a6b-4782-bef8-892e74a8040b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:49.838026 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be591a1-2a6b-4782-bef8-892e74a8040b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:49.850188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be591a1-2a6b-4782-bef8-892e74a8040b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:49.851693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be591a1-2a6b-4782-bef8-892e74a8040b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:49.869110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-7be591a1-2a6b-4782-bef8-892e74a8040b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:49.869110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:49.869110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:49.869110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:49.869110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:49.869110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:49.869110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:49.869110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:49.869110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:49.869110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:49.869110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:49.869110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:49.869110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:49.869110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:49.869110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:49.870187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:49.870187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:49.870187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:49.870187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:49.870187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:49.870187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:49.870187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:49.870187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:49.870187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:49.870187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:49.870187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:49.870187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:49.870187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:49.870187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:49.870187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:49.870187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:49.870187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:49.871202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:49.871202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:49.871202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:49.871202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:49.871202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 649/1278] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:49.878879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22107327-61e8-4a80-8f47-f1466e337079 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:49.991632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22107327-61e8-4a80-8f47-f1466e337079 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:50.028940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d5813-69f0-4db8-945a-927cabe92d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:50.029601 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d5813-69f0-4db8-945a-927cabe92d0f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:50.029881 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d5813-69f0-4db8-945a-927cabe92d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:50.030152 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d5813-69f0-4db8-945a-927cabe92d0f None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:50.030871 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d5813-69f0-4db8-945a-927cabe92d0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:50.031175 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d5813-69f0-4db8-945a-927cabe92d0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:50.032584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d5813-69f0-4db8-945a-927cabe92d0f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:50.044630 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 630/1279] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:50.053007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be591a1-2a6b-4782-bef8-892e74a8040b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:50.086043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:50.159064 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be591a1-2a6b-4782-bef8-892e74a8040b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:50.204546 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d913c6-e492-4ca0-8ab7-67f4ca212c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:50.205349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d913c6-e492-4ca0-8ab7-67f4ca212c73 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:50.205633 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d913c6-e492-4ca0-8ab7-67f4ca212c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:50.205911 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d913c6-e492-4ca0-8ab7-67f4ca212c73 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:50.213717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d913c6-e492-4ca0-8ab7-67f4ca212c73 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:50.215177 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d913c6-e492-4ca0-8ab7-67f4ca212c73 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:50.229738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-07d913c6-e492-4ca0-8ab7-67f4ca212c73 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:50.229738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:50.229738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:50.229738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:50.229738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:50.229738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:50.229738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:50.229738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:50.229738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:50.229738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:50.229738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:50.229738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:50.229738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:06:50.229738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:06:50.229738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:06:50.229738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:06:50.230847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:50.230847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:50.230847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:06:50.230847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:50.230847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:06:50.230847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:50.230847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:50.230847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:50.230847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:50.230847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:50.230847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:50.230847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:50.230847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:50.230847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:50.230847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:50.230847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:50.230847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:50.230847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:50.231857 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:50.231857 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:50.231857 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:06:50.231857 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:06:50.231857 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:06:50.231857 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:06:50.231857 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:06:50.231857 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:06:50.231857 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:50.231857 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:50.231857 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 650/1280] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:50.237321 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d5813-69f0-4db8-945a-927cabe92d0f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:50.243238 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:50.355344 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d5813-69f0-4db8-945a-927cabe92d0f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:50.392660 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952c92b-6b0d-4bb9-b7e1-c660db8dc978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:50.393333 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952c92b-6b0d-4bb9-b7e1-c660db8dc978 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:50.393576 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952c92b-6b0d-4bb9-b7e1-c660db8dc978 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:50.393877 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952c92b-6b0d-4bb9-b7e1-c660db8dc978 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:50.394714 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952c92b-6b0d-4bb9-b7e1-c660db8dc978 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:50.395023 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952c92b-6b0d-4bb9-b7e1-c660db8dc978 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:50.396501 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952c92b-6b0d-4bb9-b7e1-c660db8dc978 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:50.414330 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 631/1281] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Jun 27 21:06:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:50.420446 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d913c6-e492-4ca0-8ab7-67f4ca212c73 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:50.514658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:50.526159 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d913c6-e492-4ca0-8ab7-67f4ca212c73 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:50.561676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e435ce-73e4-493b-92f5-a0750c352f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:50.562313 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e435ce-73e4-493b-92f5-a0750c352f49 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:50.562593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e435ce-73e4-493b-92f5-a0750c352f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:50.562864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e435ce-73e4-493b-92f5-a0750c352f49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:50.634246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e435ce-73e4-493b-92f5-a0750c352f49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:50.636126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e435ce-73e4-493b-92f5-a0750c352f49 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:50.727391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:50.740155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 651/1282] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Jun 27 21:06:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:06:50.746271 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952c92b-6b0d-4bb9-b7e1-c660db8dc978 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:50.814608 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:50.856543 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952c92b-6b0d-4bb9-b7e1-c660db8dc978 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:50.898990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0cfc54-6719-44e9-abfd-6fbd9b114033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:50.899825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cfc54-6719-44e9-abfd-6fbd9b114033 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:50.900106 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cfc54-6719-44e9-abfd-6fbd9b114033 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:50.900376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cfc54-6719-44e9-abfd-6fbd9b114033 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:50.912925 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0cfc54-6719-44e9-abfd-6fbd9b114033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:50.914397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0cfc54-6719-44e9-abfd-6fbd9b114033 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:50.932961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-9d0cfc54-6719-44e9-abfd-6fbd9b114033 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:50.932961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:50.932961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:50.932961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:50.932961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:50.932961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:50.932961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:50.932961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:50.932961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:50.932961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:50.932961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:50.932961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:50.932961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:50.932961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:50.932961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:50.933996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:50.933996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:50.933996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:50.933996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:50.933996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:50.933996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:50.933996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:50.933996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:50.933996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:50.933996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:50.933996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:50.933996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:50.933996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:50.933996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:50.933996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:50.933996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:50.933996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:50.935032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:50.935032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:50.935032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:50.935032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:06:50.935032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 632/1283] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:50.942398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e435ce-73e4-493b-92f5-a0750c352f49 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:51.042620 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e435ce-73e4-493b-92f5-a0750c352f49 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:51.077771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccdf3a6-3679-413e-8abb-a952fb686f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:51.078427 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdf3a6-3679-413e-8abb-a952fb686f33 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:51.078702 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdf3a6-3679-413e-8abb-a952fb686f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:51.078974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdf3a6-3679-413e-8abb-a952fb686f33 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:51.079719 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccdf3a6-3679-413e-8abb-a952fb686f33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:51.080025 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccdf3a6-3679-413e-8abb-a952fb686f33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:51.081405 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccdf3a6-3679-413e-8abb-a952fb686f33 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:51.092986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 652/1284] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:51.098970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0cfc54-6719-44e9-abfd-6fbd9b114033 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:51.206130 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0cfc54-6719-44e9-abfd-6fbd9b114033 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:51.220982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:51.243692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a667c-7cac-4dbf-aa46-0363b7b3067e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:51.244453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a667c-7cac-4dbf-aa46-0363b7b3067e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:51.244734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a667c-7cac-4dbf-aa46-0363b7b3067e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:51.245011 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a667c-7cac-4dbf-aa46-0363b7b3067e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:51.252019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a667c-7cac-4dbf-aa46-0363b7b3067e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:51.253478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a667c-7cac-4dbf-aa46-0363b7b3067e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:51.267509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-030a667c-7cac-4dbf-aa46-0363b7b3067e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:51.267509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:51.267509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:51.267509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:51.267509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:51.267509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:51.267509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:51.267509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:51.267509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:51.267509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:51.267509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:51.267509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:51.267509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:06:51.267509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:06:51.267509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:06:51.268587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:06:51.268587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:51.268587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:51.268587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:51.268587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:51.268587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:51.268587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:51.268587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:51.268587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:51.268587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:51.268587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:51.268587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:51.268587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:51.268587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:51.268587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:51.268587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:51.268587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:51.269811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:06:51.269811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:06:51.269811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:06:51.269811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:06:51.269811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:06:51.269811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:06:51.269811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:51.269811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Jun 27 21:06:51.269811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 633/1285] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Jun 27 21:06:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:06:51.274748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccdf3a6-3679-413e-8abb-a952fb686f33 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:51.328945 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:51.386777 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccdf3a6-3679-413e-8abb-a952fb686f33 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:51.427568 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5e3a5-bc81-4749-943c-cf66bebc7137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:51.428263 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5e3a5-bc81-4749-943c-cf66bebc7137 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:51.428548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5e3a5-bc81-4749-943c-cf66bebc7137 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:51.428825 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5e3a5-bc81-4749-943c-cf66bebc7137 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:51.429619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e5e3a5-bc81-4749-943c-cf66bebc7137 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:51.429930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e5e3a5-bc81-4749-943c-cf66bebc7137 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:51.431444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e5e3a5-bc81-4749-943c-cf66bebc7137 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:51.446991 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 653/1286] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Jun 27 21:06:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:51.457166 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a667c-7cac-4dbf-aa46-0363b7b3067e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:51.570766 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a667c-7cac-4dbf-aa46-0363b7b3067e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:51.606389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74584054-2a22-402b-9e19-156f78357288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:51.607038 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74584054-2a22-402b-9e19-156f78357288 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:51.607314 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74584054-2a22-402b-9e19-156f78357288 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:51.607581 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74584054-2a22-402b-9e19-156f78357288 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:51.608443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74584054-2a22-402b-9e19-156f78357288 None admin] RBAC: Adding query filter params (role.id=5c3c095bb53943eca95c4e049cd8cbf5, scope.project.id=dfc29b021d1646819137c89e93f33601, user.id=1f5fa9b0b0314ee9bb7e16b8464307b1) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:51.608755 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74584054-2a22-402b-9e19-156f78357288 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:51.610192 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74584054-2a22-402b-9e19-156f78357288 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:51.616149 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 634/1287] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Jun 27 21:06:51 2022] GET /identity/v3/role_assignments?user.id=1f5fa9b0b0314ee9bb7e16b8464307b1&scope.project.id=dfc29b021d1646819137c89e93f33601&role.id=5c3c095bb53943eca95c4e049cd8cbf5 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:53.655894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484716f1-18f5-4529-9990-50bfad243aa7 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:53.656491 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484716f1-18f5-4529-9990-50bfad243aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:53.656491 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484716f1-18f5-4529-9990-50bfad243aa7 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:53.657845 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 654/1288] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:06:53.664230 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdfefc-2bf0-4c0e-a407-8c4db065d468 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:53.664912 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdfefc-2bf0-4c0e-a407-8c4db065d468 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:53.665096 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdfefc-2bf0-4c0e-a407-8c4db065d468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:53.689176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-57bdfefc-2bf0-4c0e-a407-8c4db065d468 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:06:53.690369 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-57bdfefc-2bf0-4c0e-a407-8c4db065d468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:06:53.722661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:53.772246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 635/1289] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:06:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:06:53.796413 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484716f1-18f5-4529-9990-50bfad243aa7 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:53.905487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484716f1-18f5-4529-9990-50bfad243aa7 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:53.916300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:53.940813 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c6d3e-04df-4c8a-9a53-5888f78ef2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:53.941613 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c6d3e-04df-4c8a-9a53-5888f78ef2bc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:53.941889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c6d3e-04df-4c8a-9a53-5888f78ef2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:53.942163 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c6d3e-04df-4c8a-9a53-5888f78ef2bc None admin] PATH_INFO: `/v3/services/placement` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:53.942815 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c6d3e-04df-4c8a-9a53-5888f78ef2bc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:53.944247 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c6d3e-04df-4c8a-9a53-5888f78ef2bc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:53.956423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f96c6d3e-04df-4c8a-9a53-5888f78ef2bc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 21:06:53.956423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:53.956423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:06:53.956423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:06:53.956423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:06:53.956423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:06:53.956423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:06:53.956423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:06:53.956423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 21:06:53.956423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:53.956423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:06:53.956423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:53.956423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:53.956423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:53.956423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:53.956423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:53.956423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:53.957537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:53.957537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:53.957537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:53.957537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:53.957537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:53.957537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:53.957537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:06:53.957537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:06:53.957537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:06:53.957537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:06:53.957537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:53.957537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:53.957537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:53.957537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:53.957537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:53.957537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:53.958560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:53.958560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:53.958560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:53.958560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:53.958560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:53.958560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:53.958560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:53.958560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:53.958560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:53.958560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:53.958560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:06:53.958560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:06:53.958560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 21:06:53.958560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:53.958560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 655/1290] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Jun 27 21:06:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:53.964057 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bdfefc-2bf0-4c0e-a407-8c4db065d468 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:54.070424 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bdfefc-2bf0-4c0e-a407-8c4db065d468 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:54.114853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766dbf23-abf3-4646-b2e9-ad06e5ea40bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:54.115650 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766dbf23-abf3-4646-b2e9-ad06e5ea40bd None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:54.115960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766dbf23-abf3-4646-b2e9-ad06e5ea40bd None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:54.116376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766dbf23-abf3-4646-b2e9-ad06e5ea40bd None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:54.117518 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766dbf23-abf3-4646-b2e9-ad06e5ea40bd None admin] RBAC: Adding query filter params (name=placement) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:54.118009 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766dbf23-abf3-4646-b2e9-ad06e5ea40bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:54.119874 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766dbf23-abf3-4646-b2e9-ad06e5ea40bd None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:54.125483 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 636/1291] 10.222.0.27 () {56 vars in 1116 bytes} [Mon Jun 27 21:06:53 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:54.132167 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c6d3e-04df-4c8a-9a53-5888f78ef2bc None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:54.252732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c6d3e-04df-4c8a-9a53-5888f78ef2bc None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:54.293492 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2246405-8c51-4a23-99c8-79963bd604f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:54.294151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2246405-8c51-4a23-99c8-79963bd604f4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:54.294426 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2246405-8c51-4a23-99c8-79963bd604f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:54.294695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2246405-8c51-4a23-99c8-79963bd604f4 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:54.295393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2246405-8c51-4a23-99c8-79963bd604f4 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:54.295725 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2246405-8c51-4a23-99c8-79963bd604f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:54.297110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2246405-8c51-4a23-99c8-79963bd604f4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:54.302317 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 656/1292] 10.222.0.27 () {56 vars in 1116 bytes} [Mon Jun 27 21:06:54 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:56.273797 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594441c-73c1-4c4b-8f93-54f21fe0502a None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:56.274666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594441c-73c1-4c4b-8f93-54f21fe0502a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:56.274666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594441c-73c1-4c4b-8f93-54f21fe0502a None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:56.276963 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 637/1293] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:06:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:06:56.283763 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b60404-8198-4969-854c-a817d1950048 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:56.284481 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b60404-8198-4969-854c-a817d1950048 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:56.284673 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b60404-8198-4969-854c-a817d1950048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:56.305269 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-53b60404-8198-4969-854c-a817d1950048 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:06:56.306486 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-53b60404-8198-4969-854c-a817d1950048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:06:56.338573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:56.378417 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 657/1294] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:06:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:06:56.401437 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594441c-73c1-4c4b-8f93-54f21fe0502a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:56.438731 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:56.522949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594441c-73c1-4c4b-8f93-54f21fe0502a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:56.558706 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457dd2ce-195b-4a3d-afca-2e897e0c70ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:56.559306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457dd2ce-195b-4a3d-afca-2e897e0c70ca None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:56.559585 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457dd2ce-195b-4a3d-afca-2e897e0c70ca None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:56.559892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457dd2ce-195b-4a3d-afca-2e897e0c70ca None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:56.560614 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457dd2ce-195b-4a3d-afca-2e897e0c70ca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:56.561990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457dd2ce-195b-4a3d-afca-2e897e0c70ca None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:56.578491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 638/1295] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Jun 27 21:06:56 2022] POST /identity/v3/services => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:06:58.639643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea0ef0-4a06-47ff-a6ea-a116e22aad25 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:58.640246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea0ef0-4a06-47ff-a6ea-a116e22aad25 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:58.640246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea0ef0-4a06-47ff-a6ea-a116e22aad25 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:58.641601 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 658/1296] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:06:58.648409 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba965a-8599-4f08-8d9c-2dc3b8e6d676 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:58.649091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba965a-8599-4f08-8d9c-2dc3b8e6d676 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:58.649243 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba965a-8599-4f08-8d9c-2dc3b8e6d676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:58.668806 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-eeba965a-8599-4f08-8d9c-2dc3b8e6d676 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:06:58.669989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-eeba965a-8599-4f08-8d9c-2dc3b8e6d676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:06:58.767449 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 639/1297] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:06:58 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:06:58.794952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea0ef0-4a06-47ff-a6ea-a116e22aad25 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:58.908214 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea0ef0-4a06-47ff-a6ea-a116e22aad25 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:58.943109 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c2a263-8755-4b1d-b81f-91d42f6dbc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:58.943944 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2a263-8755-4b1d-b81f-91d42f6dbc6e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:58.944226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2a263-8755-4b1d-b81f-91d42f6dbc6e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:58.944500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2a263-8755-4b1d-b81f-91d42f6dbc6e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:58.945152 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c2a263-8755-4b1d-b81f-91d42f6dbc6e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:58.946543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c2a263-8755-4b1d-b81f-91d42f6dbc6e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:58.958794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-09c2a263-8755-4b1d-b81f-91d42f6dbc6e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 21:06:58.958794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:58.958794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:06:58.958794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:06:58.958794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:06:58.958794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:06:58.958794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:06:58.958794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:06:58.958794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 21:06:58.958794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:58.958794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:06:58.958794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:58.958794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:58.958794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:58.958794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:58.958794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:58.958794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:58.959960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:58.959960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:58.959960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:58.959960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:58.959960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:58.959960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:58.959960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:06:58.959960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:06:58.959960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:06:58.959960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:06:58.959960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:58.959960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:58.959960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:58.959960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:58.959960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:58.959960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:58.960826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:58.960826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:58.960826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:58.960826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:58.960826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:58.960826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:58.960826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:58.960826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:58.960826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:58.960826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:58.960826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:06:58.960826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:06:58.960826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 21:06:58.960826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:06:58.960826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 659/1298] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Jun 27 21:06:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:58.966601 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba965a-8599-4f08-8d9c-2dc3b8e6d676 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:58.985636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:59.078826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba965a-8599-4f08-8d9c-2dc3b8e6d676 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:59.116725 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b632d9-5a04-4268-81c3-7108482cd9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:59.117389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b632d9-5a04-4268-81c3-7108482cd9ff None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:59.117666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b632d9-5a04-4268-81c3-7108482cd9ff None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:59.117945 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b632d9-5a04-4268-81c3-7108482cd9ff None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:59.118647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b632d9-5a04-4268-81c3-7108482cd9ff None admin] RBAC: Adding query filter params (name=placement) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:59.118980 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b632d9-5a04-4268-81c3-7108482cd9ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:59.120381 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b632d9-5a04-4268-81c3-7108482cd9ff None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:59.125556 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 640/1299] 10.222.0.27 () {56 vars in 1116 bytes} [Mon Jun 27 21:06:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:06:59.131941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c2a263-8755-4b1d-b81f-91d42f6dbc6e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:59.168062 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:59.248658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c2a263-8755-4b1d-b81f-91d42f6dbc6e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:59.289503 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8fb58-9b19-4248-aa31-cebd8bf219ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:59.290164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8fb58-9b19-4248-aa31-cebd8bf219ac None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:59.290488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8fb58-9b19-4248-aa31-cebd8bf219ac None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:59.290762 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8fb58-9b19-4248-aa31-cebd8bf219ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:59.291551 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8fb58-9b19-4248-aa31-cebd8bf219ac None admin] RBAC: Adding query filter params (interface=public, service_id=20d4dc18998947058535d46e71b3028c, region_id=RegionOne) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:59.291890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8fb58-9b19-4248-aa31-cebd8bf219ac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:59.293296 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8fb58-9b19-4248-aa31-cebd8bf219ac None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:59.298496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 660/1300] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Jun 27 21:06:59 2022] GET /identity/v3/endpoints?service_id=20d4dc18998947058535d46e71b3028c&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:06:59.307682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b632d9-5a04-4268-81c3-7108482cd9ff None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:59.400088 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:59.420052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b632d9-5a04-4268-81c3-7108482cd9ff None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:59.454620 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784d849-3594-4a0c-97fb-647e07641a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:59.455277 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784d849-3594-4a0c-97fb-647e07641a4e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:59.455552 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784d849-3594-4a0c-97fb-647e07641a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:59.455855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784d849-3594-4a0c-97fb-647e07641a4e None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:59.456533 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784d849-3594-4a0c-97fb-647e07641a4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:59.457907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784d849-3594-4a0c-97fb-647e07641a4e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:59.466946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 641/1301] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Jun 27 21:06:59 2022] GET /identity/v3/services => generated 2191 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:07:01.510268 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b765d-fc2f-4c85-b4d0-a0e0d70eaca4 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:01.511180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b765d-fc2f-4c85-b4d0-a0e0d70eaca4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:01.511180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b765d-fc2f-4c85-b4d0-a0e0d70eaca4 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:01.513463 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 661/1302] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:07:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:07:01.518896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522696c-e44f-4ab4-9523-179e2dc96781 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:01.519336 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522696c-e44f-4ab4-9523-179e2dc96781 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:01.519441 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522696c-e44f-4ab4-9523-179e2dc96781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:01.538554 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d522696c-e44f-4ab4-9523-179e2dc96781 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:07:01.539780 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d522696c-e44f-4ab4-9523-179e2dc96781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:07:01.611155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 642/1303] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:07:01 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:07:01.637395 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b765d-fc2f-4c85-b4d0-a0e0d70eaca4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:07:01.747222 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:07:01.755118 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b765d-fc2f-4c85-b4d0-a0e0d70eaca4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:07:01.791144 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f9ee4-98a6-4be6-935b-6f7d28eddb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:07:01.791999 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f9ee4-98a6-4be6-935b-6f7d28eddb67 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:01.792281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f9ee4-98a6-4be6-935b-6f7d28eddb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:01.792547 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f9ee4-98a6-4be6-935b-6f7d28eddb67 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:01.793200 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f9ee4-98a6-4be6-935b-6f7d28eddb67 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:07:01.794608 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f9ee4-98a6-4be6-935b-6f7d28eddb67 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:07:01.806748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-d88f9ee4-98a6-4be6-935b-6f7d28eddb67 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 21:07:01.806748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:07:01.806748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:07:01.806748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:07:01.806748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:07:01.806748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:07:01.806748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:07:01.806748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:07:01.806748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 21:07:01.806748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:07:01.806748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:07:01.806748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:07:01.806748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:07:01.806748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:07:01.806748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:07:01.806748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:07:01.806748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:07:01.807892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:07:01.807892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:07:01.807892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:07:01.807892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:07:01.807892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:07:01.807892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:07:01.807892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:07:01.807892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:07:01.807892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:07:01.807892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:07:01.807892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:07:01.807892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:07:01.807892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:07:01.807892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:07:01.807892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:07:01.807892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:07:01.808894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:07:01.808894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 27 21:07:01.808894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:07:01.808894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:07:01.808894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:07:01.808894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:07:01.808894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:07:01.808894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:07:01.808894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:07:01.808894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:07:01.808894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:07:01.808894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:07:01.808894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 21:07:01.808894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 27 21:07:01.808894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 662/1304] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Jun 27 21:07:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:07:01.816273 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522696c-e44f-4ab4-9523-179e2dc96781 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:07:01.897479 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:07:01.922099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522696c-e44f-4ab4-9523-179e2dc96781 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:07:01.957273 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef58368-497d-45ea-a49d-5c39ffdf6b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:07:01.957936 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef58368-497d-45ea-a49d-5c39ffdf6b73 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:01.958225 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef58368-497d-45ea-a49d-5c39ffdf6b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:01.958502 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef58368-497d-45ea-a49d-5c39ffdf6b73 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:01.959214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef58368-497d-45ea-a49d-5c39ffdf6b73 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:07:01.959523 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef58368-497d-45ea-a49d-5c39ffdf6b73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:07:01.960934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef58368-497d-45ea-a49d-5c39ffdf6b73 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:07:01.966134 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 643/1305] 10.222.0.27 () {56 vars in 1116 bytes} [Mon Jun 27 21:07:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:07:01.972223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f9ee4-98a6-4be6-935b-6f7d28eddb67 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:07:02.081629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f9ee4-98a6-4be6-935b-6f7d28eddb67 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:07:02.116561 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d832f-0d93-43b6-97f0-924518492322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:07:02.117212 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d832f-0d93-43b6-97f0-924518492322 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:02.117489 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d832f-0d93-43b6-97f0-924518492322 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:02.117761 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d832f-0d93-43b6-97f0-924518492322 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:02.118510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d832f-0d93-43b6-97f0-924518492322 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:07:02.119920 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d832f-0d93-43b6-97f0-924518492322 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:07:02.146531 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 663/1306] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Jun 27 21:07:01 2022] POST /identity/v3/endpoints => generated 331 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:07:11.963563 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=30 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:07:12.138895 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:04.999666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f1a63-9508-4acf-b565-58fb97e18f4c None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:05.000159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f1a63-9508-4acf-b565-58fb97e18f4c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:05.000243 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f1a63-9508-4acf-b565-58fb97e18f4c None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:05.001807 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 644/1307] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:05.007069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c5dfc-6101-469b-88aa-47a861231647 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:05.007526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c5dfc-6101-469b-88aa-47a861231647 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:05.007630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c5dfc-6101-469b-88aa-47a861231647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:05.083193 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-283c5dfc-6101-469b-88aa-47a861231647 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:05.084413 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-283c5dfc-6101-469b-88aa-47a861231647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:05.203799 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 664/1308] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:05.263343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448297e8-2348-448c-b12b-f04a56228200 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:05.263343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448297e8-2348-448c-b12b-f04a56228200 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:05.263343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448297e8-2348-448c-b12b-f04a56228200 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:05.266749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 645/1309] 10.222.0.27 () {54 vars in 964 bytes} [Mon Jun 27 21:08:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:05.271278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c5631-949a-4e6c-9068-812f1f9d342e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:05.271570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c5631-949a-4e6c-9068-812f1f9d342e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:05.271880 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c5631-949a-4e6c-9068-812f1f9d342e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:05.360349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c87c5631-949a-4e6c-9068-812f1f9d342e None None] MFA Rules not processed for user `9c7ce29db6b14176a4c9be436bcddc6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:05.361045 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c87c5631-949a-4e6c-9068-812f1f9d342e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:05.382559 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:05.459354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 665/1310] 10.222.0.27 () {58 vars in 1061 bytes} [Mon Jun 27 21:08:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:08:05.468144 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448297e8-2348-448c-b12b-f04a56228200 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:05.470179 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-448297e8-2348-448c-b12b-f04a56228200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:05.596920 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448297e8-2348-448c-b12b-f04a56228200 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:05.678633 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dae80a-7ddb-4f6f-b4c7-bf777b2ffc72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:05.679410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dae80a-7ddb-4f6f-b4c7-bf777b2ffc72 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:05.679725 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dae80a-7ddb-4f6f-b4c7-bf777b2ffc72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:05.679964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dae80a-7ddb-4f6f-b4c7-bf777b2ffc72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:05.681656 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d6dae80a-7ddb-4f6f-b4c7-bf777b2ffc72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:05.745288 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:05.806343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dae80a-7ddb-4f6f-b4c7-bf777b2ffc72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:05.807962 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dae80a-7ddb-4f6f-b4c7-bf777b2ffc72 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:05.829462 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 646/1311] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:08:08.161607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27972551-d343-46a8-9352-5a658bd59e51 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:08.162200 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27972551-d343-46a8-9352-5a658bd59e51 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:08.162200 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27972551-d343-46a8-9352-5a658bd59e51 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:08.163632 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 666/1312] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:08.169150 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db209831-b695-484c-8d51-9cecc78b1e2b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:08.169624 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db209831-b695-484c-8d51-9cecc78b1e2b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:08.169731 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db209831-b695-484c-8d51-9cecc78b1e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:08.258823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-db209831-b695-484c-8d51-9cecc78b1e2b None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:08.260130 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-db209831-b695-484c-8d51-9cecc78b1e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:08.379642 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 647/1313] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:08.413725 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27972551-d343-46a8-9352-5a658bd59e51 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:08.415333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-27972551-d343-46a8-9352-5a658bd59e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:08.529175 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27972551-d343-46a8-9352-5a658bd59e51 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:08.608711 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:08.612104 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbff9a-1630-4c1e-88d9-55ae77d74cdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:08.612903 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbff9a-1630-4c1e-88d9-55ae77d74cdb service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:08.613283 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbff9a-1630-4c1e-88d9-55ae77d74cdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:08.613678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbff9a-1630-4c1e-88d9-55ae77d74cdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:08.616505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-debbff9a-1630-4c1e-88d9-55ae77d74cdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:08.738643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debbff9a-1630-4c1e-88d9-55ae77d74cdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:08.740321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debbff9a-1630-4c1e-88d9-55ae77d74cdb service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:08.760095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 667/1314] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:11.105746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190ab2b-1de1-4379-879d-05e30bbe7489 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:11.106306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190ab2b-1de1-4379-879d-05e30bbe7489 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:11.106306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190ab2b-1de1-4379-879d-05e30bbe7489 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:11.107719 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 648/1315] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:11.113103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dafa2-14ea-42ba-a5e2-091d0e2e2836 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:11.113565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dafa2-14ea-42ba-a5e2-091d0e2e2836 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:11.113658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dafa2-14ea-42ba-a5e2-091d0e2e2836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:11.182789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8c0dafa2-14ea-42ba-a5e2-091d0e2e2836 None None] MFA Rules not processed for user `a393cbda710c4f2ca8a06ed49b1c5e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:11.184088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8c0dafa2-14ea-42ba-a5e2-091d0e2e2836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:11.240584 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:11.294032 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 668/1316] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:11 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:11.333635 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190ab2b-1de1-4379-879d-05e30bbe7489 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:11.335206 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8190ab2b-1de1-4379-879d-05e30bbe7489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:11.345669 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:11.448964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190ab2b-1de1-4379-879d-05e30bbe7489 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:11.523627 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fae826-ccb3-4e85-9c3d-c9b457076148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:11.524317 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fae826-ccb3-4e85-9c3d-c9b457076148 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:11.524980 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fae826-ccb3-4e85-9c3d-c9b457076148 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:11.525249 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fae826-ccb3-4e85-9c3d-c9b457076148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:11.526948 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-49fae826-ccb3-4e85-9c3d-c9b457076148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:11.593456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:11.661195 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fae826-ccb3-4e85-9c3d-c9b457076148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:11.662662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fae826-ccb3-4e85-9c3d-c9b457076148 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:11.683014 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 649/1317] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:11 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:14.295582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ddfe9-2bfa-4fa7-92b9-d46180765ed3 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:14.296209 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ddfe9-2bfa-4fa7-92b9-d46180765ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:14.296209 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ddfe9-2bfa-4fa7-92b9-d46180765ed3 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:14.297569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 669/1318] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:14.302899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491587a2-d00c-480d-9095-7481b9522cc9 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:14.303317 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491587a2-d00c-480d-9095-7481b9522cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:14.303426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491587a2-d00c-480d-9095-7481b9522cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:14.373150 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-491587a2-d00c-480d-9095-7481b9522cc9 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:14.374370 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-491587a2-d00c-480d-9095-7481b9522cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:14.474984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 650/1319] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:14.521753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ddfe9-2bfa-4fa7-92b9-d46180765ed3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:14.523340 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-826ddfe9-2bfa-4fa7-92b9-d46180765ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:14.632376 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ddfe9-2bfa-4fa7-92b9-d46180765ed3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:14.710603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8a38b-a049-441f-849d-41a842224a68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:14.711254 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8a38b-a049-441f-849d-41a842224a68 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:14.711492 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8a38b-a049-441f-849d-41a842224a68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:14.711829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8a38b-a049-441f-849d-41a842224a68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:14.713505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a6f8a38b-a049-441f-849d-41a842224a68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:14.721129 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:14.829602 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f8a38b-a049-441f-849d-41a842224a68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:14.831074 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f8a38b-a049-441f-849d-41a842224a68 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:14.849696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 670/1320] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:17.538285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741066cf-2fc6-4107-b897-8f39edef803e None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:17.538880 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741066cf-2fc6-4107-b897-8f39edef803e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:17.538880 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741066cf-2fc6-4107-b897-8f39edef803e None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:17.540267 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 651/1321] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:17.545487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe53e4-d292-440e-bda9-b424089d2684 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:17.545912 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe53e4-d292-440e-bda9-b424089d2684 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:17.546157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe53e4-d292-440e-bda9-b424089d2684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:17.612947 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-44fe53e4-d292-440e-bda9-b424089d2684 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:17.614257 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-44fe53e4-d292-440e-bda9-b424089d2684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:17.678495 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:17.725272 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 671/1322] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:17.761271 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741066cf-2fc6-4107-b897-8f39edef803e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:17.762898 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-741066cf-2fc6-4107-b897-8f39edef803e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:17.784893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:17.880437 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741066cf-2fc6-4107-b897-8f39edef803e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:17.958439 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd76649-da7b-4908-804d-7805e43acd85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:17.959104 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd76649-da7b-4908-804d-7805e43acd85 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:17.959380 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd76649-da7b-4908-804d-7805e43acd85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:17.959651 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd76649-da7b-4908-804d-7805e43acd85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:17.961332 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9bd76649-da7b-4908-804d-7805e43acd85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:18.038942 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:18.079464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd76649-da7b-4908-804d-7805e43acd85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:18.080998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd76649-da7b-4908-804d-7805e43acd85 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:18.102289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 652/1323] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:20.140486 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08d1a9-0275-49d5-9985-b0a426dd70f1 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:20.141021 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08d1a9-0275-49d5-9985-b0a426dd70f1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:20.141021 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08d1a9-0275-49d5-9985-b0a426dd70f1 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:20.142298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 672/1324] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:20.147847 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79eb052-6e03-4086-9503-928658726800 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:20.148312 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79eb052-6e03-4086-9503-928658726800 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:20.148408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79eb052-6e03-4086-9503-928658726800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:20.214040 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e79eb052-6e03-4086-9503-928658726800 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:20.215255 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e79eb052-6e03-4086-9503-928658726800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:20.315638 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 653/1325] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:20.350715 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08d1a9-0275-49d5-9985-b0a426dd70f1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:20.352359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4e08d1a9-0275-49d5-9985-b0a426dd70f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:20.461498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08d1a9-0275-49d5-9985-b0a426dd70f1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:20.541569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87430c94-7ef8-459f-8c2e-1c9dcecc2693 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:20.542242 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87430c94-7ef8-459f-8c2e-1c9dcecc2693 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:20.542523 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87430c94-7ef8-459f-8c2e-1c9dcecc2693 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:20.542795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87430c94-7ef8-459f-8c2e-1c9dcecc2693 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:20.544631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-87430c94-7ef8-459f-8c2e-1c9dcecc2693 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:20.566325 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:20.663570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87430c94-7ef8-459f-8c2e-1c9dcecc2693 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:20.665061 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87430c94-7ef8-459f-8c2e-1c9dcecc2693 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:20.684331 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 673/1326] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:22.733210 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccc649-4055-44ab-aeb9-2c32303fc627 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:22.733757 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccc649-4055-44ab-aeb9-2c32303fc627 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:22.733757 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccc649-4055-44ab-aeb9-2c32303fc627 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:22.735138 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 654/1327] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:22.740974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56852a-3a0f-4d7e-a615-58f2a314a0d3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:22.741462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56852a-3a0f-4d7e-a615-58f2a314a0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:22.741598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56852a-3a0f-4d7e-a615-58f2a314a0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:22.809487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ee56852a-3a0f-4d7e-a615-58f2a314a0d3 None None] MFA Rules not processed for user `a393cbda710c4f2ca8a06ed49b1c5e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:22.810703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ee56852a-3a0f-4d7e-a615-58f2a314a0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:22.875997 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:22.911239 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 674/1328] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:22 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:22.963406 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94658a-59eb-40ce-9ac7-fb35a9bdb5df None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:22.963823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94658a-59eb-40ce-9ac7-fb35a9bdb5df None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:22.964163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94658a-59eb-40ce-9ac7-fb35a9bdb5df None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:22.965658 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 655/1329] 10.222.0.27 () {54 vars in 964 bytes} [Mon Jun 27 21:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:22.971599 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2392bd-d300-47e9-8ba6-32d3e6fc41f2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:22.971916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2392bd-d300-47e9-8ba6-32d3e6fc41f2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:22.972188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2392bd-d300-47e9-8ba6-32d3e6fc41f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:23.052360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6e2392bd-d300-47e9-8ba6-32d3e6fc41f2 None None] MFA Rules not processed for user `9c7ce29db6b14176a4c9be436bcddc6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:23.053570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6e2392bd-d300-47e9-8ba6-32d3e6fc41f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:23.134696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 675/1330] 10.222.0.27 () {58 vars in 1061 bytes} [Mon Jun 27 21:08:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:23.143801 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94658a-59eb-40ce-9ac7-fb35a9bdb5df None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:23.145383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7a94658a-59eb-40ce-9ac7-fb35a9bdb5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:23.177535 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:23.252222 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94658a-59eb-40ce-9ac7-fb35a9bdb5df None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:23.349061 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e7b32-2982-4fd7-9e6c-6f740109b05e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:23.349708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e7b32-2982-4fd7-9e6c-6f740109b05e service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:23.349981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e7b32-2982-4fd7-9e6c-6f740109b05e service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:23.350256 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e7b32-2982-4fd7-9e6c-6f740109b05e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:23.351987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-bc6e7b32-2982-4fd7-9e6c-6f740109b05e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:23.433538 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:23.468246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e7b32-2982-4fd7-9e6c-6f740109b05e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:23.469729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e7b32-2982-4fd7-9e6c-6f740109b05e service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:23.490846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 656/1331] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:23 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:26.121351 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b6196-74e5-48cf-99d8-663be7a3d646 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:26.121935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b6196-74e5-48cf-99d8-663be7a3d646 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:26.121935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b6196-74e5-48cf-99d8-663be7a3d646 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:26.123292 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 676/1332] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:26.129450 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea577ad-95aa-4349-aa19-a33da01724c8 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:26.129890 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea577ad-95aa-4349-aa19-a33da01724c8 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:26.129998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea577ad-95aa-4349-aa19-a33da01724c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:26.206682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6ea577ad-95aa-4349-aa19-a33da01724c8 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:26.207977 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6ea577ad-95aa-4349-aa19-a33da01724c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:26.326723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 657/1333] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:26.363913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b6196-74e5-48cf-99d8-663be7a3d646 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:26.365486 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6c7b6196-74e5-48cf-99d8-663be7a3d646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:26.445649 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:26.483958 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b6196-74e5-48cf-99d8-663be7a3d646 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:26.562339 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8f069-b7e6-4288-82e1-1c79b9d158c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:26.562998 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8f069-b7e6-4288-82e1-1c79b9d158c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:26.563281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8f069-b7e6-4288-82e1-1c79b9d158c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:26.563555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8f069-b7e6-4288-82e1-1c79b9d158c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:26.565284 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e1c8f069-b7e6-4288-82e1-1c79b9d158c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:26.684779 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8f069-b7e6-4288-82e1-1c79b9d158c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:26.686145 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8f069-b7e6-4288-82e1-1c79b9d158c2 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:26.696706 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:26.705343 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 677/1334] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:28.788865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29cbf34-3562-41cb-adc3-c2eabbedb6ed None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:28.789688 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29cbf34-3562-41cb-adc3-c2eabbedb6ed None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:28.789688 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29cbf34-3562-41cb-adc3-c2eabbedb6ed None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:28.792009 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 658/1335] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:08:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:28.800784 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb939d-4e45-497c-9c42-6ad7ee00719f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:28.801192 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb939d-4e45-497c-9c42-6ad7ee00719f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:28.801338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb939d-4e45-497c-9c42-6ad7ee00719f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:28.868974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-77fb939d-4e45-497c-9c42-6ad7ee00719f None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:28.870204 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-77fb939d-4e45-497c-9c42-6ad7ee00719f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:28.982300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 678/1336] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:29.028720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29cbf34-3562-41cb-adc3-c2eabbedb6ed None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:29.030289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b29cbf34-3562-41cb-adc3-c2eabbedb6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:29.075184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:29.142566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29cbf34-3562-41cb-adc3-c2eabbedb6ed None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:29.219023 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453a3b8-9996-4231-a472-ce3518074e3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:29.219709 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453a3b8-9996-4231-a472-ce3518074e3e service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:29.219995 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453a3b8-9996-4231-a472-ce3518074e3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:29.220271 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453a3b8-9996-4231-a472-ce3518074e3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:29.221926 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f453a3b8-9996-4231-a472-ce3518074e3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:29.307581 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:29.341482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453a3b8-9996-4231-a472-ce3518074e3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:29.342940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453a3b8-9996-4231-a472-ce3518074e3e service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:29.361327 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 659/1337] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:31.858082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c630327-9d22-4833-aaf0-0a0f0f7b4d3d None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:31.858955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c630327-9d22-4833-aaf0-0a0f0f7b4d3d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:31.858955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c630327-9d22-4833-aaf0-0a0f0f7b4d3d None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:31.861725 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 679/1338] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:08:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:31.867119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf875a-8463-4f49-a650-d71ba1c7c2d9 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:31.867540 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf875a-8463-4f49-a650-d71ba1c7c2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:31.867744 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf875a-8463-4f49-a650-d71ba1c7c2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:31.935804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d3bf875a-8463-4f49-a650-d71ba1c7c2d9 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:31.937759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d3bf875a-8463-4f49-a650-d71ba1c7c2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:32.048233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 660/1339] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:31 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:32.084230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c630327-9d22-4833-aaf0-0a0f0f7b4d3d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:32.085781 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3c630327-9d22-4833-aaf0-0a0f0f7b4d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:32.189006 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:32.217582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c630327-9d22-4833-aaf0-0a0f0f7b4d3d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:32.293930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088c2fe-3965-4aaa-a128-50cbf8dd1501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:32.294585 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088c2fe-3965-4aaa-a128-50cbf8dd1501 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:32.294864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088c2fe-3965-4aaa-a128-50cbf8dd1501 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:32.295138 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088c2fe-3965-4aaa-a128-50cbf8dd1501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:32.296846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5088c2fe-3965-4aaa-a128-50cbf8dd1501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:32.428090 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088c2fe-3965-4aaa-a128-50cbf8dd1501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:32.429562 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088c2fe-3965-4aaa-a128-50cbf8dd1501 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:32.450800 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 680/1340] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:36.487009 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16e4c6-7f89-4a31-bb70-853ea591cdae None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:36.487565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16e4c6-7f89-4a31-bb70-853ea591cdae None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:36.487699 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16e4c6-7f89-4a31-bb70-853ea591cdae None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:36.490267 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 661/1341] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:08:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:36.497074 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f55b7-b5c6-401b-9e90-c0072629a508 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:36.497548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f55b7-b5c6-401b-9e90-c0072629a508 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:36.497634 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f55b7-b5c6-401b-9e90-c0072629a508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:36.504306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:36.572834 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-1f0f55b7-b5c6-401b-9e90-c0072629a508 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:36.574007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1f0f55b7-b5c6-401b-9e90-c0072629a508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:36.681334 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 681/1342] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:36.720247 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16e4c6-7f89-4a31-bb70-853ea591cdae None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:36.722786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4f16e4c6-7f89-4a31-bb70-853ea591cdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:36.785801 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:36.835781 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16e4c6-7f89-4a31-bb70-853ea591cdae None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:36.839509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98688c-d139-4e7b-b588-c4a64dea0b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:36.840221 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98688c-d139-4e7b-b588-c4a64dea0b98 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:36.840506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98688c-d139-4e7b-b588-c4a64dea0b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:36.840780 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98688c-d139-4e7b-b588-c4a64dea0b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:36.850114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98688c-d139-4e7b-b588-c4a64dea0b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:36.851623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98688c-d139-4e7b-b588-c4a64dea0b98 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:36.861070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 662/1343] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:39.539921 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64989247-cd1f-4cf8-a1ee-5eb8463f6827 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:39.540495 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64989247-cd1f-4cf8-a1ee-5eb8463f6827 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:39.540495 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64989247-cd1f-4cf8-a1ee-5eb8463f6827 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:39.541864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 682/1344] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:39.547546 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b821c-9109-4f77-999c-0566ec6ee056 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:39.548131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b821c-9109-4f77-999c-0566ec6ee056 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:39.548131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b821c-9109-4f77-999c-0566ec6ee056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:39.580485 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-0e6b821c-9109-4f77-999c-0566ec6ee056 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:39.581815 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0e6b821c-9109-4f77-999c-0566ec6ee056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:39.673388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 663/1345] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:39 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:39.713829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64989247-cd1f-4cf8-a1ee-5eb8463f6827 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:39.715798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-64989247-cd1f-4cf8-a1ee-5eb8463f6827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:39.731086 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64989247-cd1f-4cf8-a1ee-5eb8463f6827 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:39.736158 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95851367-5a2c-4e4c-9a9d-e3742a1a6fb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:39.737013 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95851367-5a2c-4e4c-9a9d-e3742a1a6fb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:39.737443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95851367-5a2c-4e4c-9a9d-e3742a1a6fb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:39.737912 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95851367-5a2c-4e4c-9a9d-e3742a1a6fb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:39.749789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95851367-5a2c-4e4c-9a9d-e3742a1a6fb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:39.751268 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95851367-5a2c-4e4c-9a9d-e3742a1a6fb4 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:39.762259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 683/1346] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:39 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:43.727537 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511215c-41aa-4686-88d5-02969df9a487 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:43.728100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511215c-41aa-4686-88d5-02969df9a487 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:43.728206 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511215c-41aa-4686-88d5-02969df9a487 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:43.729586 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 664/1347] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:43.735262 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a081ef-4dba-420a-86ea-49073e4653eb None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:43.735730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a081ef-4dba-420a-86ea-49073e4653eb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:43.735876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a081ef-4dba-420a-86ea-49073e4653eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:43.754102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-44a081ef-4dba-420a-86ea-49073e4653eb None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:43.755324 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-44a081ef-4dba-420a-86ea-49073e4653eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:43.764470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 684/1348] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:43.802531 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511215c-41aa-4686-88d5-02969df9a487 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:43.813550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511215c-41aa-4686-88d5-02969df9a487 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:43.817039 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea05186-afca-4260-b18f-e9b6926cd15a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:43.817684 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea05186-afca-4260-b18f-e9b6926cd15a service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:43.817970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea05186-afca-4260-b18f-e9b6926cd15a service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:43.818242 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea05186-afca-4260-b18f-e9b6926cd15a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:43.828540 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea05186-afca-4260-b18f-e9b6926cd15a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:43.830020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea05186-afca-4260-b18f-e9b6926cd15a service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:43.839834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 665/1349] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:48.791116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42f553-28a9-4a54-b674-e0bba107bd38 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:48.792287 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42f553-28a9-4a54-b674-e0bba107bd38 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:48.792287 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42f553-28a9-4a54-b674-e0bba107bd38 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:48.793933 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 685/1350] 10.222.0.27 () {54 vars in 921 bytes} [Mon Jun 27 21:08:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:48.800428 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fe142-5c03-43ad-a88e-a69096bce705 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:48.800867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fe142-5c03-43ad-a88e-a69096bce705 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:48.800973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fe142-5c03-43ad-a88e-a69096bce705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:48.901639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-f99fe142-5c03-43ad-a88e-a69096bce705 None None] MFA Rules not processed for user `1f5fa9b0b0314ee9bb7e16b8464307b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:48.903053 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f99fe142-5c03-43ad-a88e-a69096bce705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:48.934679 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:48.995184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 666/1351] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Jun 27 21:08:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:08:53.256199 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b3f39-2031-46a8-9d85-03c409b22f78 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:53.257112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b3f39-2031-46a8-9d85-03c409b22f78 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:53.257562 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b3f39-2031-46a8-9d85-03c409b22f78 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:53.259193 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 686/1352] 10.222.0.27 () {54 vars in 921 bytes} [Mon Jun 27 21:08:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:53.265087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f288d3-409b-4acf-93e4-ea7de7ad9663 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:53.265644 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f288d3-409b-4acf-93e4-ea7de7ad9663 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:53.265644 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f288d3-409b-4acf-93e4-ea7de7ad9663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:53.285716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-19f288d3-409b-4acf-93e4-ea7de7ad9663 None None] MFA Rules not processed for user `1f5fa9b0b0314ee9bb7e16b8464307b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:53.287017 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-19f288d3-409b-4acf-93e4-ea7de7ad9663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:53.295974 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 667/1353] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Jun 27 21:08:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:53.752313 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=72 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:54.600085 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef58a12-c78d-44b4-aeba-573970ac2b59 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:54.600826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef58a12-c78d-44b4-aeba-573970ac2b59 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:54.600826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef58a12-c78d-44b4-aeba-573970ac2b59 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:54.602247 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 687/1354] 10.222.0.27 () {54 vars in 921 bytes} [Mon Jun 27 21:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:54.607136 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5ec5a-2807-4dc7-9f9d-f0b4c63ea4ab None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:54.607652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5ec5a-2807-4dc7-9f9d-f0b4c63ea4ab None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:54.607779 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5ec5a-2807-4dc7-9f9d-f0b4c63ea4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:54.626081 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-37d5ec5a-2807-4dc7-9f9d-f0b4c63ea4ab None None] MFA Rules not processed for user `1f5fa9b0b0314ee9bb7e16b8464307b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:54.627864 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-37d5ec5a-2807-4dc7-9f9d-f0b4c63ea4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:54.637869 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 668/1355] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Jun 27 21:08:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:08:58.237332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77e6d3-ec49-4224-a1fa-4fed1d4d4599 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:58.237883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77e6d3-ec49-4224-a1fa-4fed1d4d4599 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:58.237883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77e6d3-ec49-4224-a1fa-4fed1d4d4599 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:58.240668 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 688/1356] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:08:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:58.244508 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169096f-02e1-4f9e-abf2-01d9cfcb2be3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:58.245285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169096f-02e1-4f9e-abf2-01d9cfcb2be3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:58.245706 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169096f-02e1-4f9e-abf2-01d9cfcb2be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:58.272294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e169096f-02e1-4f9e-abf2-01d9cfcb2be3 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:58.274121 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e169096f-02e1-4f9e-abf2-01d9cfcb2be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:58.286532 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 669/1357] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:58.345683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f8170-1e80-44db-92e8-37c1ea97a26e None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:58.346251 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f8170-1e80-44db-92e8-37c1ea97a26e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:58.346749 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f8170-1e80-44db-92e8-37c1ea97a26e None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:58.349243 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 689/1358] 10.222.0.27 () {54 vars in 961 bytes} [Mon Jun 27 21:08:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:58.359132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28f5be-7493-4ddd-855c-709c6f618490 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:58.359436 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28f5be-7493-4ddd-855c-709c6f618490 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:58.360250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28f5be-7493-4ddd-855c-709c6f618490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:58.434080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-fc28f5be-7493-4ddd-855c-709c6f618490 None None] MFA Rules not processed for user `8d7fe01ef64f46f6a0e77d5945219b33`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:58.435556 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fc28f5be-7493-4ddd-855c-709c6f618490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:58.531021 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 670/1359] 10.222.0.27 () {58 vars in 1058 bytes} [Mon Jun 27 21:08:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:08:58.532238 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f8170-1e80-44db-92e8-37c1ea97a26e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:58.549622 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f8170-1e80-44db-92e8-37c1ea97a26e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:58.559647 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cde4b7-715c-47d6-8dac-fb473849c1f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:58.560559 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cde4b7-715c-47d6-8dac-fb473849c1f3 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:58.561040 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cde4b7-715c-47d6-8dac-fb473849c1f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:58.561453 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cde4b7-715c-47d6-8dac-fb473849c1f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:58.571933 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cde4b7-715c-47d6-8dac-fb473849c1f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:58.573975 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cde4b7-715c-47d6-8dac-fb473849c1f3 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:58.585011 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 690/1360] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:00.826038 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7090c-ff46-43bc-b02b-ea8c95e73f50 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:00.826610 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7090c-ff46-43bc-b02b-ea8c95e73f50 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:00.826610 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7090c-ff46-43bc-b02b-ea8c95e73f50 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:00.828555 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 671/1361] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:09:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:00.833377 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125aa43-0827-476b-b217-1a4b21d64e12 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:00.833836 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125aa43-0827-476b-b217-1a4b21d64e12 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:00.833915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125aa43-0827-476b-b217-1a4b21d64e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:00.852517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7125aa43-0827-476b-b217-1a4b21d64e12 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:00.853704 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7125aa43-0827-476b-b217-1a4b21d64e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:00.862022 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 691/1362] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:09:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:00.906213 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7090c-ff46-43bc-b02b-ea8c95e73f50 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:00.921294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7090c-ff46-43bc-b02b-ea8c95e73f50 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:00.927042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9991ca-5f4f-4e0a-9705-2f30d20bcd55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:00.927808 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9991ca-5f4f-4e0a-9705-2f30d20bcd55 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:00.928164 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9991ca-5f4f-4e0a-9705-2f30d20bcd55 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:00.928500 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9991ca-5f4f-4e0a-9705-2f30d20bcd55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:00.936991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9991ca-5f4f-4e0a-9705-2f30d20bcd55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:00.938708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9991ca-5f4f-4e0a-9705-2f30d20bcd55 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:00.949289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 672/1363] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:03.304607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db81bca-d8f5-4083-ab55-f9a7ff5360db None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:03.305139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db81bca-d8f5-4083-ab55-f9a7ff5360db None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:03.305139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db81bca-d8f5-4083-ab55-f9a7ff5360db None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:03.306590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 692/1364] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:03.312080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9ac2f-1904-4de9-8017-194f709a14ca None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:03.312505 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9ac2f-1904-4de9-8017-194f709a14ca None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:03.312597 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9ac2f-1904-4de9-8017-194f709a14ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:03.331513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b1c9ac2f-1904-4de9-8017-194f709a14ca None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:03.333116 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b1c9ac2f-1904-4de9-8017-194f709a14ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:03.352158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 673/1365] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:09:03.388460 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403fa18-9a89-4702-b652-80411ca8e317 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:03.388758 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403fa18-9a89-4702-b652-80411ca8e317 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:03.389024 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403fa18-9a89-4702-b652-80411ca8e317 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:03.390413 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 693/1366] 10.222.0.27 () {54 vars in 961 bytes} [Mon Jun 27 21:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:03.396049 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a06db0-60fe-45c1-b871-6a44acadc23c None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:03.396338 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a06db0-60fe-45c1-b871-6a44acadc23c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:03.396619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a06db0-60fe-45c1-b871-6a44acadc23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:03.415255 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-77a06db0-60fe-45c1-b871-6a44acadc23c None None] MFA Rules not processed for user `8d7fe01ef64f46f6a0e77d5945219b33`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:03.416592 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-77a06db0-60fe-45c1-b871-6a44acadc23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:03.426956 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 674/1367] 10.222.0.27 () {58 vars in 1058 bytes} [Mon Jun 27 21:09:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:09:03.435302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403fa18-9a89-4702-b652-80411ca8e317 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:03.450975 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403fa18-9a89-4702-b652-80411ca8e317 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:03.456992 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de839f-f043-481a-a90c-14424f61086b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:03.457671 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de839f-f043-481a-a90c-14424f61086b service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:03.457874 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de839f-f043-481a-a90c-14424f61086b service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:03.458148 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de839f-f043-481a-a90c-14424f61086b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:03.466813 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de839f-f043-481a-a90c-14424f61086b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:03.468475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de839f-f043-481a-a90c-14424f61086b service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:03.478084 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 694/1368] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:09:05.550846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1078b-45d5-4b53-b69a-cf521caff097 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:05.551397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1078b-45d5-4b53-b69a-cf521caff097 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:05.551397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1078b-45d5-4b53-b69a-cf521caff097 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:05.552817 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 675/1369] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:05.558090 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347bb34-7548-43f3-98a6-f17638251400 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:05.558516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347bb34-7548-43f3-98a6-f17638251400 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:05.558647 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347bb34-7548-43f3-98a6-f17638251400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:05.576436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9347bb34-7548-43f3-98a6-f17638251400 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:05.577642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9347bb34-7548-43f3-98a6-f17638251400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:05.585687 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 695/1370] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:05.628947 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1078b-45d5-4b53-b69a-cf521caff097 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:05.644593 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1078b-45d5-4b53-b69a-cf521caff097 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:05.650822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c4a44-91e5-44d7-8f63-faf4b245f92a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:05.651503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c4a44-91e5-44d7-8f63-faf4b245f92a service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:05.651797 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c4a44-91e5-44d7-8f63-faf4b245f92a service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:05.652242 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c4a44-91e5-44d7-8f63-faf4b245f92a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:05.662261 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c4a44-91e5-44d7-8f63-faf4b245f92a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:05.663932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c4a44-91e5-44d7-8f63-faf4b245f92a service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:05.674949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 676/1371] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:06.324816 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2e1e0-c69e-49d7-91da-1fa217f27729 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:06.325077 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2e1e0-c69e-49d7-91da-1fa217f27729 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:06.325342 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2e1e0-c69e-49d7-91da-1fa217f27729 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:06.326918 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 696/1372] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:06.332670 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234de2b-2940-424b-9f39-49c848895a1d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:06.332923 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234de2b-2940-424b-9f39-49c848895a1d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:06.333235 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234de2b-2940-424b-9f39-49c848895a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:06.351648 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-4234de2b-2940-424b-9f39-49c848895a1d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:06.352875 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4234de2b-2940-424b-9f39-49c848895a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:06.361504 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 677/1373] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:06.667648 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdb8a2-344a-4cfb-83da-f3da44de9300 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:06.668523 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdb8a2-344a-4cfb-83da-f3da44de9300 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:06.668877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdb8a2-344a-4cfb-83da-f3da44de9300 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:06.670427 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 697/1374] 10.222.0.27 () {54 vars in 963 bytes} [Mon Jun 27 21:09:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:06.675763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7038a7-7e73-423a-863f-e691283dde41 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:06.676243 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7038a7-7e73-423a-863f-e691283dde41 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:06.676344 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7038a7-7e73-423a-863f-e691283dde41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:06.683329 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:09:06.762648 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-3f7038a7-7e73-423a-863f-e691283dde41 None None] MFA Rules not processed for user `16fda36445d9415aae4df74f31af0601`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:06.764025 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3f7038a7-7e73-423a-863f-e691283dde41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:06.835135 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 678/1375] 10.222.0.27 () {58 vars in 1060 bytes} [Mon Jun 27 21:09:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:06.843138 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdb8a2-344a-4cfb-83da-f3da44de9300 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:06.855905 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdb8a2-344a-4cfb-83da-f3da44de9300 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:06.860142 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48099f0-526d-4824-a249-c81d4937fa04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:06.861133 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48099f0-526d-4824-a249-c81d4937fa04 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:06.861740 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48099f0-526d-4824-a249-c81d4937fa04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:06.862202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48099f0-526d-4824-a249-c81d4937fa04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:06.873869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48099f0-526d-4824-a249-c81d4937fa04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:06.875501 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48099f0-526d-4824-a249-c81d4937fa04 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:06.887461 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 698/1376] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:07.735329 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51eada6-db3c-413a-b372-2e996c72cec3 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:07.735901 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51eada6-db3c-413a-b372-2e996c72cec3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:07.735983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51eada6-db3c-413a-b372-2e996c72cec3 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:07.738494 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 679/1377] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:09:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:07.743743 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48059b56-ce78-4850-b637-f8ddcc59899e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:07.743743 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48059b56-ce78-4850-b637-f8ddcc59899e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:07.744348 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48059b56-ce78-4850-b637-f8ddcc59899e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:07.763870 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-48059b56-ce78-4850-b637-f8ddcc59899e None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:07.765229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-48059b56-ce78-4850-b637-f8ddcc59899e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:07.776085 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 699/1378] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:09:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:07.817382 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51eada6-db3c-413a-b372-2e996c72cec3 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:07.831868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51eada6-db3c-413a-b372-2e996c72cec3 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:07.837347 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ceb11-4028-4552-8621-f0357d5eaf85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:07.837993 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ceb11-4028-4552-8621-f0357d5eaf85 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:07.838276 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ceb11-4028-4552-8621-f0357d5eaf85 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:07.838557 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ceb11-4028-4552-8621-f0357d5eaf85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:07.847530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ceb11-4028-4552-8621-f0357d5eaf85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:07.849099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ceb11-4028-4552-8621-f0357d5eaf85 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:07.858871 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 680/1379] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:09.918348 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dcf9e-ae10-4cce-a039-df3e6fe1af78 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:09.919197 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dcf9e-ae10-4cce-a039-df3e6fe1af78 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:09.919661 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dcf9e-ae10-4cce-a039-df3e6fe1af78 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:09.921250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 700/1380] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:09:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:09.927176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21cd07-d5ad-4805-a263-800a803b74d9 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:09.927672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21cd07-d5ad-4805-a263-800a803b74d9 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:09.927965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21cd07-d5ad-4805-a263-800a803b74d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:09.946463 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-9b21cd07-d5ad-4805-a263-800a803b74d9 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:09.947700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9b21cd07-d5ad-4805-a263-800a803b74d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:09.956125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 681/1381] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:09:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:10.002304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dcf9e-ae10-4cce-a039-df3e6fe1af78 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:10.018063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dcf9e-ae10-4cce-a039-df3e6fe1af78 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:10.028183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a773b-e7d7-4959-aacd-f53a7cb04cbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:10.028957 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a773b-e7d7-4959-aacd-f53a7cb04cbf service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:10.029277 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a773b-e7d7-4959-aacd-f53a7cb04cbf service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:10.029609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a773b-e7d7-4959-aacd-f53a7cb04cbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:10.038650 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a773b-e7d7-4959-aacd-f53a7cb04cbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:10.040314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a773b-e7d7-4959-aacd-f53a7cb04cbf service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:10.049768 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 701/1382] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:12.118848 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedafcb2-2428-4165-9d49-5e678044db30 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:12.119413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedafcb2-2428-4165-9d49-5e678044db30 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:12.119413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedafcb2-2428-4165-9d49-5e678044db30 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:12.121622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 682/1383] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:09:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:12.128642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf961deb-d7ce-465f-9ad3-9502e2546328 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:12.129131 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf961deb-d7ce-465f-9ad3-9502e2546328 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:12.129578 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf961deb-d7ce-465f-9ad3-9502e2546328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:12.152055 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-cf961deb-d7ce-465f-9ad3-9502e2546328 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:12.153284 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cf961deb-d7ce-465f-9ad3-9502e2546328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:12.162561 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 702/1384] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:09:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:12.205184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedafcb2-2428-4165-9d49-5e678044db30 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:12.221004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedafcb2-2428-4165-9d49-5e678044db30 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:12.226864 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2752af7-4914-4448-b023-9506df9fa3bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:12.227819 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2752af7-4914-4448-b023-9506df9fa3bd service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:12.228251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2752af7-4914-4448-b023-9506df9fa3bd service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:12.228674 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2752af7-4914-4448-b023-9506df9fa3bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:12.239227 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2752af7-4914-4448-b023-9506df9fa3bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:12.240979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2752af7-4914-4448-b023-9506df9fa3bd service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:12.251836 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 683/1385] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:14.303289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5f2c8-429f-4c2c-8c2e-8577a3036b35 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:14.304068 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5f2c8-429f-4c2c-8c2e-8577a3036b35 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:14.304068 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5f2c8-429f-4c2c-8c2e-8577a3036b35 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:14.306508 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 703/1386] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:09:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:14.312051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fce6ea-4e5a-40ed-a402-03153fd99c7b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:14.312461 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fce6ea-4e5a-40ed-a402-03153fd99c7b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:14.312784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fce6ea-4e5a-40ed-a402-03153fd99c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:14.332622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-49fce6ea-4e5a-40ed-a402-03153fd99c7b None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:14.333829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-49fce6ea-4e5a-40ed-a402-03153fd99c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:14.343848 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 684/1387] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:14.388733 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5f2c8-429f-4c2c-8c2e-8577a3036b35 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:14.406755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5f2c8-429f-4c2c-8c2e-8577a3036b35 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:14.412261 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7efd14-ed5d-4c3a-a080-f9672ac40f4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:14.415741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7efd14-ed5d-4c3a-a080-f9672ac40f4b service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:14.416178 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7efd14-ed5d-4c3a-a080-f9672ac40f4b service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:14.416548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7efd14-ed5d-4c3a-a080-f9672ac40f4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:14.426989 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7efd14-ed5d-4c3a-a080-f9672ac40f4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:14.429022 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7efd14-ed5d-4c3a-a080-f9672ac40f4b service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:14.440013 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 704/1388] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:16.896312 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c69c6c-2cc4-4f0b-923e-7def4a4aaf49 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:16.896970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c69c6c-2cc4-4f0b-923e-7def4a4aaf49 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:16.896970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c69c6c-2cc4-4f0b-923e-7def4a4aaf49 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:16.898711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 685/1389] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:09:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:16.905445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cea2f3-f1ac-4567-be8d-00bdb099c07c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:16.906111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cea2f3-f1ac-4567-be8d-00bdb099c07c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:16.906111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cea2f3-f1ac-4567-be8d-00bdb099c07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:16.927459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-93cea2f3-f1ac-4567-be8d-00bdb099c07c None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:16.928793 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-93cea2f3-f1ac-4567-be8d-00bdb099c07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:16.938913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 705/1390] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:09:16.988176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c69c6c-2cc4-4f0b-923e-7def4a4aaf49 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:17.006122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c69c6c-2cc4-4f0b-923e-7def4a4aaf49 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:17.011869 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bca3e1-8dc1-443f-ae07-5b073919a8c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:17.012542 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bca3e1-8dc1-443f-ae07-5b073919a8c4 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:17.012826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bca3e1-8dc1-443f-ae07-5b073919a8c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:17.013115 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bca3e1-8dc1-443f-ae07-5b073919a8c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:17.021955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bca3e1-8dc1-443f-ae07-5b073919a8c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:17.023981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bca3e1-8dc1-443f-ae07-5b073919a8c4 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:17.034283 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 686/1391] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:19.164476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df113871-a814-466a-b776-dee60a99d629 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:19.165134 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df113871-a814-466a-b776-dee60a99d629 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:19.165134 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df113871-a814-466a-b776-dee60a99d629 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:19.166534 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 706/1392] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:19.172103 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1dfdc-1303-4546-99ff-eb6312ec3428 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:19.172790 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1dfdc-1303-4546-99ff-eb6312ec3428 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:19.172970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1dfdc-1303-4546-99ff-eb6312ec3428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:19.198269 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b3a1dfdc-1303-4546-99ff-eb6312ec3428 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:19.200022 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b3a1dfdc-1303-4546-99ff-eb6312ec3428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:19.211919 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 687/1393] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:19.257870 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df113871-a814-466a-b776-dee60a99d629 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:19.278950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df113871-a814-466a-b776-dee60a99d629 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:19.286215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f417c-6887-4537-8f6c-c1a086acf202 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:19.286933 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f417c-6887-4537-8f6c-c1a086acf202 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:19.287252 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f417c-6887-4537-8f6c-c1a086acf202 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:19.287531 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f417c-6887-4537-8f6c-c1a086acf202 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:19.296507 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f417c-6887-4537-8f6c-c1a086acf202 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:19.298251 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f417c-6887-4537-8f6c-c1a086acf202 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:19.308142 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 707/1394] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:21.365409 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84a95e-3154-409d-988a-ecd3a19c7154 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:21.366011 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84a95e-3154-409d-988a-ecd3a19c7154 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:21.366011 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84a95e-3154-409d-988a-ecd3a19c7154 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:21.367497 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 688/1395] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:21.372820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0183b-308a-4d35-bbb2-04eb5a18b7c0 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:21.373266 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0183b-308a-4d35-bbb2-04eb5a18b7c0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:21.373363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0183b-308a-4d35-bbb2-04eb5a18b7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:21.392569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-00c0183b-308a-4d35-bbb2-04eb5a18b7c0 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:21.393811 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-00c0183b-308a-4d35-bbb2-04eb5a18b7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:21.402537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 708/1396] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:21.449021 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84a95e-3154-409d-988a-ecd3a19c7154 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:21.465455 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84a95e-3154-409d-988a-ecd3a19c7154 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:21.472289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf22a8-7e4f-46a2-bc32-fb67507e7b82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:21.472956 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf22a8-7e4f-46a2-bc32-fb67507e7b82 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:21.473270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf22a8-7e4f-46a2-bc32-fb67507e7b82 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:21.473543 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf22a8-7e4f-46a2-bc32-fb67507e7b82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:21.482709 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf22a8-7e4f-46a2-bc32-fb67507e7b82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:21.484359 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf22a8-7e4f-46a2-bc32-fb67507e7b82 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:21.496377 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 689/1397] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:21.939706 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff78553-8780-4c13-87b7-93b775fe831b None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:21.939998 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff78553-8780-4c13-87b7-93b775fe831b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:21.940273 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff78553-8780-4c13-87b7-93b775fe831b None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:21.941691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 709/1398] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:21.947009 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6f59a-40fa-4c8c-8e34-75e127d311b7 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:21.947399 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6f59a-40fa-4c8c-8e34-75e127d311b7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:21.947775 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6f59a-40fa-4c8c-8e34-75e127d311b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:21.966224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-04d6f59a-40fa-4c8c-8e34-75e127d311b7 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:21.967443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-04d6f59a-40fa-4c8c-8e34-75e127d311b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:21.976470 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 690/1399] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:22.031470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68197804-7406-4432-9c07-678ceeaa4243 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:22.031792 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68197804-7406-4432-9c07-678ceeaa4243 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:22.032090 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68197804-7406-4432-9c07-678ceeaa4243 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:22.033692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 710/1400] 10.222.0.27 () {54 vars in 962 bytes} [Mon Jun 27 21:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:22.038754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90567c8-f064-49c9-82a2-9031078b33b9 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:22.039045 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90567c8-f064-49c9-82a2-9031078b33b9 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:22.039322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90567c8-f064-49c9-82a2-9031078b33b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:22.089602 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e90567c8-f064-49c9-82a2-9031078b33b9 None None] MFA Rules not processed for user `b43db1fd0b494a3f8588b498b585f247`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:22.090821 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e90567c8-f064-49c9-82a2-9031078b33b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:22.123060 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:09:22.152165 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 691/1401] 10.222.0.27 () {58 vars in 1059 bytes} [Mon Jun 27 21:09:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:22.160289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68197804-7406-4432-9c07-678ceeaa4243 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:22.171556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68197804-7406-4432-9c07-678ceeaa4243 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:22.175378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87892b6-da78-4170-83ba-efd93a10e133 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b43db1fd0b494a3f8588b498b585f247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:22.176052 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87892b6-da78-4170-83ba-efd93a10e133 service glance] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:22.176327 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87892b6-da78-4170-83ba-efd93a10e133 service glance] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:22.176600 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87892b6-da78-4170-83ba-efd93a10e133 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:22.185853 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87892b6-da78-4170-83ba-efd93a10e133 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:22.187297 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87892b6-da78-4170-83ba-efd93a10e133 service glance] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:22.196939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 711/1402] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:22.916123 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66a9a3-e1ee-4082-a0ee-8e1134ddb777 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:22.916637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66a9a3-e1ee-4082-a0ee-8e1134ddb777 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:22.916637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66a9a3-e1ee-4082-a0ee-8e1134ddb777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:22.966767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-fd66a9a3-e1ee-4082-a0ee-8e1134ddb777 None None] MFA Rules not processed for user `e802da0b078e457da466a9a7b0f15085`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:22.968232 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fd66a9a3-e1ee-4082-a0ee-8e1134ddb777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:23.039627 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 692/1403] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Jun 27 21:09:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:23.083488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84248a2f-525e-4b56-8418-5d5c52d47942 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:23.083994 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84248a2f-525e-4b56-8418-5d5c52d47942 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:23.084096 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84248a2f-525e-4b56-8418-5d5c52d47942 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:23.085490 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 712/1404] 10.222.0.27 () {54 vars in 942 bytes} [Mon Jun 27 21:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:23.091805 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f206e3a-3f4d-43ab-bd50-bafc8341615f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:23.092129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f206e3a-3f4d-43ab-bd50-bafc8341615f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:23.092473 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f206e3a-3f4d-43ab-bd50-bafc8341615f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:23.142071 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-4f206e3a-3f4d-43ab-bd50-bafc8341615f None None] MFA Rules not processed for user `f28acb0322114aebafcf62bdf9a346df`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:23.143737 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4f206e3a-3f4d-43ab-bd50-bafc8341615f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:23.223947 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 693/1405] 10.222.0.27 () {58 vars in 1039 bytes} [Mon Jun 27 21:09:23 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:23.233181 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84248a2f-525e-4b56-8418-5d5c52d47942 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:23.247440 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84248a2f-525e-4b56-8418-5d5c52d47942 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:23.253102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16e6bf-0800-40ab-8491-a1909b9e29cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28acb0322114aebafcf62bdf9a346df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:23.253745 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16e6bf-0800-40ab-8491-a1909b9e29cc service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:23.254019 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16e6bf-0800-40ab-8491-a1909b9e29cc service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:23.254286 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16e6bf-0800-40ab-8491-a1909b9e29cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:23.262285 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16e6bf-0800-40ab-8491-a1909b9e29cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:23.263904 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16e6bf-0800-40ab-8491-a1909b9e29cc service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:23.272119 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 713/1406] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Jun 27 21:09:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:09:23.527070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68602c8-3c23-4e46-a4f5-7b6c034d69d0 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:23.527350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68602c8-3c23-4e46-a4f5-7b6c034d69d0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:23.527649 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68602c8-3c23-4e46-a4f5-7b6c034d69d0 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:23.531643 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 694/1407] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:23.534773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187a059-6ef3-461a-a86e-399fa7fa1cb3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:23.535067 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187a059-6ef3-461a-a86e-399fa7fa1cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:23.535344 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187a059-6ef3-461a-a86e-399fa7fa1cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:23.554613 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f187a059-6ef3-461a-a86e-399fa7fa1cb3 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:23.555980 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f187a059-6ef3-461a-a86e-399fa7fa1cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:23.572400 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 714/1408] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:09:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:09:23.610164 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68602c8-3c23-4e46-a4f5-7b6c034d69d0 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:23.627183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68602c8-3c23-4e46-a4f5-7b6c034d69d0 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:23.632392 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947f75e-e6c7-4993-8e12-3e7979f594f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:23.633069 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947f75e-e6c7-4993-8e12-3e7979f594f8 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:23.633351 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947f75e-e6c7-4993-8e12-3e7979f594f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:23.633620 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947f75e-e6c7-4993-8e12-3e7979f594f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:23.642506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947f75e-e6c7-4993-8e12-3e7979f594f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:23.644137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947f75e-e6c7-4993-8e12-3e7979f594f8 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:23.649618 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:09:23.664566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 695/1409] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:33.549250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=69 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:09:45.241623 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb91a91-c8d7-43d8-952f-1c01ad3af6fa None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:45.242328 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb91a91-c8d7-43d8-952f-1c01ad3af6fa None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:45.242328 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb91a91-c8d7-43d8-952f-1c01ad3af6fa None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:45.243557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 715/1410] 10.222.0.27 () {54 vars in 918 bytes} [Mon Jun 27 21:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:45.249062 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b4a98-6e2e-45ff-bebf-2ab2a6f88aa5 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:45.249479 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b4a98-6e2e-45ff-bebf-2ab2a6f88aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:45.249617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b4a98-6e2e-45ff-bebf-2ab2a6f88aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:45.268698 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-597b4a98-6e2e-45ff-bebf-2ab2a6f88aa5 None None] MFA Rules not processed for user `1f5fa9b0b0314ee9bb7e16b8464307b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:45.269953 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-597b4a98-6e2e-45ff-bebf-2ab2a6f88aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:45.279535 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 696/1411] 10.222.0.27 () {58 vars in 1015 bytes} [Mon Jun 27 21:09:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:45.294132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bb10b-d75b-4ea5-aeb2-a6d5e4c21c9b None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:45.294424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bb10b-d75b-4ea5-aeb2-a6d5e4c21c9b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:45.294699 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bb10b-d75b-4ea5-aeb2-a6d5e4c21c9b None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:45.296112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 716/1412] 10.222.0.27 () {54 vars in 918 bytes} [Mon Jun 27 21:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:45.300877 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e1e27-ba50-407b-be2e-a69043617267 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:45.301199 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e1e27-ba50-407b-be2e-a69043617267 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:45.301476 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e1e27-ba50-407b-be2e-a69043617267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:45.319297 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b10e1e27-ba50-407b-be2e-a69043617267 None None] MFA Rules not processed for user `8d7fe01ef64f46f6a0e77d5945219b33`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:45.320533 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b10e1e27-ba50-407b-be2e-a69043617267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:45.331187 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 697/1413] 10.222.0.27 () {58 vars in 1015 bytes} [Mon Jun 27 21:09:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:09:47.822034 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f886b9-0f85-4111-9f6f-2bb71ebb94ee None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:47.822591 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f886b9-0f85-4111-9f6f-2bb71ebb94ee None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:47.822591 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f886b9-0f85-4111-9f6f-2bb71ebb94ee None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:47.824808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 717/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:09:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:47.833679 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b1c84-a9a2-4b4e-b310-2f2f1082ddf4 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:47.834093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b1c84-a9a2-4b4e-b310-2f2f1082ddf4 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:47.834275 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b1c84-a9a2-4b4e-b310-2f2f1082ddf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:47.857306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b57b1c84-a9a2-4b4e-b310-2f2f1082ddf4 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:47.858518 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b57b1c84-a9a2-4b4e-b310-2f2f1082ddf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:47.867572 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 698/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:09:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:47.910951 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f886b9-0f85-4111-9f6f-2bb71ebb94ee None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:47.926808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f886b9-0f85-4111-9f6f-2bb71ebb94ee None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:47.933313 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa3db7-87a7-43ef-a86a-450d34fe6243 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:47.933958 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa3db7-87a7-43ef-a86a-450d34fe6243 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:47.934198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa3db7-87a7-43ef-a86a-450d34fe6243 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:47.934508 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa3db7-87a7-43ef-a86a-450d34fe6243 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:47.943247 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa3db7-87a7-43ef-a86a-450d34fe6243 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:47.944882 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa3db7-87a7-43ef-a86a-450d34fe6243 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:47.954437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 718/1416] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:48.845597 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d235a77-b5f3-4d07-b85a-cb2d64a4efbd None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:48.846185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d235a77-b5f3-4d07-b85a-cb2d64a4efbd None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:48.846185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d235a77-b5f3-4d07-b85a-cb2d64a4efbd None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:48.847539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 699/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:48.852777 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74133153-79c2-4820-9ffb-d9094b9e9c94 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:48.853247 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74133153-79c2-4820-9ffb-d9094b9e9c94 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:48.853352 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74133153-79c2-4820-9ffb-d9094b9e9c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:48.872727 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-74133153-79c2-4820-9ffb-d9094b9e9c94 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:48.873934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-74133153-79c2-4820-9ffb-d9094b9e9c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:48.882577 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 719/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:49.030775 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d235a77-b5f3-4d07-b85a-cb2d64a4efbd None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:49.057086 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d235a77-b5f3-4d07-b85a-cb2d64a4efbd None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:49.065219 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d68f2-ee2f-4247-a2c8-ca65aa819dce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:49.066022 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d68f2-ee2f-4247-a2c8-ca65aa819dce service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:49.066307 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d68f2-ee2f-4247-a2c8-ca65aa819dce service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:49.066614 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d68f2-ee2f-4247-a2c8-ca65aa819dce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:49.080160 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d68f2-ee2f-4247-a2c8-ca65aa819dce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:49.081786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d68f2-ee2f-4247-a2c8-ca65aa819dce service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:49.092148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 700/1419] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:49.957752 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b17260-ffbb-4806-883a-5d29d547fe69 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:49.958541 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b17260-ffbb-4806-883a-5d29d547fe69 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:49.958541 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b17260-ffbb-4806-883a-5d29d547fe69 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:49.960789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 720/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:09:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:49.965918 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ec5ba-39d7-44e6-8bdc-1cb44cde3804 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:49.966393 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ec5ba-39d7-44e6-8bdc-1cb44cde3804 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:49.966499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ec5ba-39d7-44e6-8bdc-1cb44cde3804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:49.987090 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e22ec5ba-39d7-44e6-8bdc-1cb44cde3804 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:49.988338 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e22ec5ba-39d7-44e6-8bdc-1cb44cde3804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:49.996614 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 701/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:50.156045 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b17260-ffbb-4806-883a-5d29d547fe69 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:50.172240 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b17260-ffbb-4806-883a-5d29d547fe69 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:50.178115 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890325a0-6e63-4ab5-bc72-16b0dd8e0d59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:50.178757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890325a0-6e63-4ab5-bc72-16b0dd8e0d59 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:50.179033 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890325a0-6e63-4ab5-bc72-16b0dd8e0d59 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:50.179303 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890325a0-6e63-4ab5-bc72-16b0dd8e0d59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:50.187970 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890325a0-6e63-4ab5-bc72-16b0dd8e0d59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:50.189751 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890325a0-6e63-4ab5-bc72-16b0dd8e0d59 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:50.199846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 721/1422] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:51.045109 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d6a6d-0654-4a92-81d9-9a5accfbf2ee None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:51.045637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d6a6d-0654-4a92-81d9-9a5accfbf2ee None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:51.045637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d6a6d-0654-4a92-81d9-9a5accfbf2ee None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:51.046967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 702/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:51.053191 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa32a3-7444-458b-a107-34d1dd0852dd None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:51.053647 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa32a3-7444-458b-a107-34d1dd0852dd None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:51.053747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa32a3-7444-458b-a107-34d1dd0852dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:51.079710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-adfa32a3-7444-458b-a107-34d1dd0852dd None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:51.080950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-adfa32a3-7444-458b-a107-34d1dd0852dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:51.090104 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 722/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:09:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:51.246257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d6a6d-0654-4a92-81d9-9a5accfbf2ee None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:51.270613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d6a6d-0654-4a92-81d9-9a5accfbf2ee None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:51.277199 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936d40c-2683-4883-955c-541eb1bd66c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:51.277845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936d40c-2683-4883-955c-541eb1bd66c1 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:51.278249 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936d40c-2683-4883-955c-541eb1bd66c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:51.278529 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936d40c-2683-4883-955c-541eb1bd66c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:51.287103 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936d40c-2683-4883-955c-541eb1bd66c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:51.288730 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936d40c-2683-4883-955c-541eb1bd66c1 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:51.298698 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 703/1425] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:52.165937 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84d360-52c7-43fe-80c4-9fff0d5e02bd None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:52.166817 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84d360-52c7-43fe-80c4-9fff0d5e02bd None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:52.166817 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84d360-52c7-43fe-80c4-9fff0d5e02bd None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:52.168900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 723/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:09:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:52.176854 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64f8cd-5b64-4deb-a657-a6577f8ea206 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:52.177321 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64f8cd-5b64-4deb-a657-a6577f8ea206 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:52.177490 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64f8cd-5b64-4deb-a657-a6577f8ea206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:52.200905 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-2d64f8cd-5b64-4deb-a657-a6577f8ea206 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:52.202126 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2d64f8cd-5b64-4deb-a657-a6577f8ea206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:52.211239 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 704/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:52.363914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84d360-52c7-43fe-80c4-9fff0d5e02bd None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:52.385557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84d360-52c7-43fe-80c4-9fff0d5e02bd None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:52.392264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7206982-4be2-4043-b608-19731dddfbc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:52.392921 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7206982-4be2-4043-b608-19731dddfbc5 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:52.393200 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7206982-4be2-4043-b608-19731dddfbc5 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:52.393475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7206982-4be2-4043-b608-19731dddfbc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:52.402638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7206982-4be2-4043-b608-19731dddfbc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:52.404326 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7206982-4be2-4043-b608-19731dddfbc5 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:52.414920 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 724/1428] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:53.209602 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5aadb-aa74-431b-a0d1-2df9a9ead479 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:53.210148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5aadb-aa74-431b-a0d1-2df9a9ead479 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:53.210148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5aadb-aa74-431b-a0d1-2df9a9ead479 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:53.212249 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 705/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:09:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:53.220594 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a7ced-0f97-4d30-a5ea-40699d6b0977 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:53.220992 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a7ced-0f97-4d30-a5ea-40699d6b0977 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:53.221232 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a7ced-0f97-4d30-a5ea-40699d6b0977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:53.241565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-864a7ced-0f97-4d30-a5ea-40699d6b0977 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:53.242805 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-864a7ced-0f97-4d30-a5ea-40699d6b0977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:53.252324 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 725/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:09:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:53.403989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5aadb-aa74-431b-a0d1-2df9a9ead479 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:53.429166 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5aadb-aa74-431b-a0d1-2df9a9ead479 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:53.435998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f6aea-eb06-446b-ab25-de007fe99d83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:53.436650 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f6aea-eb06-446b-ab25-de007fe99d83 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:53.436931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f6aea-eb06-446b-ab25-de007fe99d83 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:53.437202 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f6aea-eb06-446b-ab25-de007fe99d83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:53.446276 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f6aea-eb06-446b-ab25-de007fe99d83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:53.447929 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f6aea-eb06-446b-ab25-de007fe99d83 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:53.458225 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 706/1431] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:54.264004 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ceaa3-76a4-4c0b-9840-77d7dbac486f None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:54.264679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ceaa3-76a4-4c0b-9840-77d7dbac486f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:54.264679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ceaa3-76a4-4c0b-9840-77d7dbac486f None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:54.265864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 726/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:54.271571 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8be65c-cdb1-425c-9073-2c883c463699 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:54.272051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8be65c-cdb1-425c-9073-2c883c463699 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:54.272156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8be65c-cdb1-425c-9073-2c883c463699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:54.293236 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-9f8be65c-cdb1-425c-9073-2c883c463699 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:54.294738 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9f8be65c-cdb1-425c-9073-2c883c463699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:54.304060 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 707/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:09:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:54.470805 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ceaa3-76a4-4c0b-9840-77d7dbac486f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:54.490418 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ceaa3-76a4-4c0b-9840-77d7dbac486f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:54.501600 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61c51e-38a4-4573-87bb-8e37a0995d0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:54.502479 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61c51e-38a4-4573-87bb-8e37a0995d0f service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:54.502800 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61c51e-38a4-4573-87bb-8e37a0995d0f service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:54.503104 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61c51e-38a4-4573-87bb-8e37a0995d0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:54.512964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61c51e-38a4-4573-87bb-8e37a0995d0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:54.514583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61c51e-38a4-4573-87bb-8e37a0995d0f service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:54.524787 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 727/1434] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:55.323720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f0717-6112-4b1e-81d7-d063453e1ce7 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:55.324308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f0717-6112-4b1e-81d7-d063453e1ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:55.324308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f0717-6112-4b1e-81d7-d063453e1ce7 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:55.325648 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 708/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:55.331660 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd937ed-9dc2-4c1e-b502-80f517f88372 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:55.332101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd937ed-9dc2-4c1e-b502-80f517f88372 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:55.332300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd937ed-9dc2-4c1e-b502-80f517f88372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:55.357578 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9dd937ed-9dc2-4c1e-b502-80f517f88372 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:55.358806 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9dd937ed-9dc2-4c1e-b502-80f517f88372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:55.368320 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 728/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:55.520068 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f0717-6112-4b1e-81d7-d063453e1ce7 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:55.546694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f0717-6112-4b1e-81d7-d063453e1ce7 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:55.555115 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82105944-82ce-4022-94fb-eb2cff9e3061 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:55.555798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82105944-82ce-4022-94fb-eb2cff9e3061 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:55.556086 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82105944-82ce-4022-94fb-eb2cff9e3061 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:55.556359 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82105944-82ce-4022-94fb-eb2cff9e3061 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:55.565451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82105944-82ce-4022-94fb-eb2cff9e3061 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:55.567068 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82105944-82ce-4022-94fb-eb2cff9e3061 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:55.577499 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 709/1437] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:56.402243 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7048c-4531-44bb-8ca2-78a8f49f6939 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:56.402844 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7048c-4531-44bb-8ca2-78a8f49f6939 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:56.402844 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7048c-4531-44bb-8ca2-78a8f49f6939 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:56.404234 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 729/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:56.410416 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcec9d-e0ca-4983-aa18-d89b79990175 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:56.411151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcec9d-e0ca-4983-aa18-d89b79990175 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:56.411274 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcec9d-e0ca-4983-aa18-d89b79990175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:56.436191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a4fcec9d-e0ca-4983-aa18-d89b79990175 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:56.437414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a4fcec9d-e0ca-4983-aa18-d89b79990175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:56.446979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 710/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:09:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:56.599590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7048c-4531-44bb-8ca2-78a8f49f6939 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:56.616452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7048c-4531-44bb-8ca2-78a8f49f6939 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:56.622475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174245d-8df2-4157-9c6a-3c6c0c9038e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:56.623127 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174245d-8df2-4157-9c6a-3c6c0c9038e9 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:56.623411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174245d-8df2-4157-9c6a-3c6c0c9038e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:56.623716 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174245d-8df2-4157-9c6a-3c6c0c9038e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:56.632482 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174245d-8df2-4157-9c6a-3c6c0c9038e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:56.634091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174245d-8df2-4157-9c6a-3c6c0c9038e9 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:56.644309 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 730/1440] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:58.571921 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cd113-5cc9-4c91-a03b-1de17f1a075f None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:58.572528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cd113-5cc9-4c91-a03b-1de17f1a075f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:58.572528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cd113-5cc9-4c91-a03b-1de17f1a075f None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:58.573860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 711/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:58.579222 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0e358-5561-417a-bc8e-18a193c749c1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:58.579711 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0e358-5561-417a-bc8e-18a193c749c1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:58.579833 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0e358-5561-417a-bc8e-18a193c749c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:58.600617 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b8d0e358-5561-417a-bc8e-18a193c749c1 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:58.601806 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b8d0e358-5561-417a-bc8e-18a193c749c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:58.612159 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 731/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:58.666017 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cd113-5cc9-4c91-a03b-1de17f1a075f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:58.681602 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cd113-5cc9-4c91-a03b-1de17f1a075f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:58.686179 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fff50-8118-4827-b6ae-372a187f8663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:58.686824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fff50-8118-4827-b6ae-372a187f8663 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:58.687108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fff50-8118-4827-b6ae-372a187f8663 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:58.687380 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fff50-8118-4827-b6ae-372a187f8663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:58.696754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7fff50-8118-4827-b6ae-372a187f8663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:58.698201 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7fff50-8118-4827-b6ae-372a187f8663 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:58.708001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 712/1443] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:10:00.861209 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c9cdc-08f2-4e7d-93f3-18a0b757bebf None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:00.862098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c9cdc-08f2-4e7d-93f3-18a0b757bebf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:00.862098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c9cdc-08f2-4e7d-93f3-18a0b757bebf None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:00.864394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 732/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:10:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:00.870856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ca6fe-576a-4513-a294-d4dceab22071 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:00.871492 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ca6fe-576a-4513-a294-d4dceab22071 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:00.871632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ca6fe-576a-4513-a294-d4dceab22071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:00.899878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-589ca6fe-576a-4513-a294-d4dceab22071 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:00.901109 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-589ca6fe-576a-4513-a294-d4dceab22071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:00.910286 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 713/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:00.948760 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c9cdc-08f2-4e7d-93f3-18a0b757bebf None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:00.963235 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c9cdc-08f2-4e7d-93f3-18a0b757bebf None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:00.967433 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65ad93-f967-4a98-bda9-0f4722565cff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:00.968165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65ad93-f967-4a98-bda9-0f4722565cff service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:00.968447 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65ad93-f967-4a98-bda9-0f4722565cff service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:00.968721 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65ad93-f967-4a98-bda9-0f4722565cff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:00.980922 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65ad93-f967-4a98-bda9-0f4722565cff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:00.983135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65ad93-f967-4a98-bda9-0f4722565cff service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:00.995014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 733/1446] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:10:03.317756 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4dbd8-020b-4ab5-a0bd-adbb8fe63d58 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:03.318386 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4dbd8-020b-4ab5-a0bd-adbb8fe63d58 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:03.318386 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4dbd8-020b-4ab5-a0bd-adbb8fe63d58 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:03.319800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 714/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:03.326272 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523d0d1-f3ca-4fdb-9c89-2a4bf33f2315 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:03.327036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523d0d1-f3ca-4fdb-9c89-2a4bf33f2315 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:03.327179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523d0d1-f3ca-4fdb-9c89-2a4bf33f2315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:03.348710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9523d0d1-f3ca-4fdb-9c89-2a4bf33f2315 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:03.349948 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9523d0d1-f3ca-4fdb-9c89-2a4bf33f2315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:03.359339 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 734/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:10:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:03.400585 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4dbd8-020b-4ab5-a0bd-adbb8fe63d58 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:03.412871 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4dbd8-020b-4ab5-a0bd-adbb8fe63d58 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:03.416968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79ede6-62a8-4384-a9e4-293da77f88be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:03.417605 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79ede6-62a8-4384-a9e4-293da77f88be service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:03.417881 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79ede6-62a8-4384-a9e4-293da77f88be service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:03.418155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79ede6-62a8-4384-a9e4-293da77f88be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:03.427774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79ede6-62a8-4384-a9e4-293da77f88be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:03.429202 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79ede6-62a8-4384-a9e4-293da77f88be service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:03.438625 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 715/1449] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:10:05.593809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73495d-a6de-49a0-a834-754679242db9 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:05.594369 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73495d-a6de-49a0-a834-754679242db9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:05.594369 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73495d-a6de-49a0-a834-754679242db9 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:05.595789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 735/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:05.602155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2616b-309b-45d6-9bf5-003c5d43633a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:05.602907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2616b-309b-45d6-9bf5-003c5d43633a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:05.602907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2616b-309b-45d6-9bf5-003c5d43633a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:05.626026 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-c6f2616b-309b-45d6-9bf5-003c5d43633a None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:05.627241 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c6f2616b-309b-45d6-9bf5-003c5d43633a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:05.637084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 716/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:10:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:05.689053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73495d-a6de-49a0-a834-754679242db9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:05.701896 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73495d-a6de-49a0-a834-754679242db9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:05.706510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b0d1e-d92b-4636-9f85-24890b2c44d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:05.707147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b0d1e-d92b-4636-9f85-24890b2c44d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:05.707428 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b0d1e-d92b-4636-9f85-24890b2c44d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:05.707765 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b0d1e-d92b-4636-9f85-24890b2c44d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:05.717785 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b0d1e-d92b-4636-9f85-24890b2c44d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:05.719222 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b0d1e-d92b-4636-9f85-24890b2c44d3 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:05.729351 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 736/1452] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:10:07.838602 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186dbee-0db4-41d3-8d63-b652bed11991 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:07.839515 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186dbee-0db4-41d3-8d63-b652bed11991 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:07.839515 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186dbee-0db4-41d3-8d63-b652bed11991 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:07.841710 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 717/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:10:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:07.850095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78109d1-0951-4203-8d03-0fbb83febe11 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:07.850545 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78109d1-0951-4203-8d03-0fbb83febe11 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:07.850655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78109d1-0951-4203-8d03-0fbb83febe11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:07.873835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f78109d1-0951-4203-8d03-0fbb83febe11 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:07.875038 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f78109d1-0951-4203-8d03-0fbb83febe11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:07.884719 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 737/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:10:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:10:07.933078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186dbee-0db4-41d3-8d63-b652bed11991 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:07.944817 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186dbee-0db4-41d3-8d63-b652bed11991 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:07.948914 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a7f11-8636-47d3-ae55-a88a5e127267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:07.949550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a7f11-8636-47d3-ae55-a88a5e127267 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:07.949830 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a7f11-8636-47d3-ae55-a88a5e127267 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:07.950106 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a7f11-8636-47d3-ae55-a88a5e127267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:07.959793 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a7f11-8636-47d3-ae55-a88a5e127267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:07.961241 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a7f11-8636-47d3-ae55-a88a5e127267 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:07.971209 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 718/1455] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:10:10.027873 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5fb9d-ca6c-4314-ac85-edddce453a63 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:10.028410 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5fb9d-ca6c-4314-ac85-edddce453a63 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:10.028410 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5fb9d-ca6c-4314-ac85-edddce453a63 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:10.029809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 738/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:10.035321 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d6a73-2631-4857-a2fa-3260f13a151d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:10.035784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d6a73-2631-4857-a2fa-3260f13a151d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:10.035906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d6a73-2631-4857-a2fa-3260f13a151d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:10.054761 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-637d6a73-2631-4857-a2fa-3260f13a151d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:10.056024 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-637d6a73-2631-4857-a2fa-3260f13a151d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:10.065042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 719/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:10:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:10.102559 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5fb9d-ca6c-4314-ac85-edddce453a63 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:10.115219 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5fb9d-ca6c-4314-ac85-edddce453a63 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:10.119532 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ab1d7-b19d-414c-9afd-a03e5989d1c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:10.120221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ab1d7-b19d-414c-9afd-a03e5989d1c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:10.120530 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ab1d7-b19d-414c-9afd-a03e5989d1c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:10.120802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ab1d7-b19d-414c-9afd-a03e5989d1c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:10.130984 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ab1d7-b19d-414c-9afd-a03e5989d1c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:10.132473 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ab1d7-b19d-414c-9afd-a03e5989d1c7 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:10.142845 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 739/1458] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:10:12.284478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0a1b9-a78a-4a27-9c1a-49aacb1027bc None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:12.285347 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0a1b9-a78a-4a27-9c1a-49aacb1027bc None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:12.285347 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0a1b9-a78a-4a27-9c1a-49aacb1027bc None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:12.287556 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 720/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:10:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:12.293244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68566840-bc07-4463-87bf-ccf2a6a6b72f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:12.293696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68566840-bc07-4463-87bf-ccf2a6a6b72f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:12.293810 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68566840-bc07-4463-87bf-ccf2a6a6b72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:12.312860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-68566840-bc07-4463-87bf-ccf2a6a6b72f None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:12.314069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-68566840-bc07-4463-87bf-ccf2a6a6b72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:12.322915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 740/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:10:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:12.360488 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0a1b9-a78a-4a27-9c1a-49aacb1027bc None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:12.377346 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0a1b9-a78a-4a27-9c1a-49aacb1027bc None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:12.382351 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505f023-1951-4713-b55f-11b4f056b35b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:12.383030 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505f023-1951-4713-b55f-11b4f056b35b service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:12.383305 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505f023-1951-4713-b55f-11b4f056b35b service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:12.383578 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505f023-1951-4713-b55f-11b4f056b35b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:12.393462 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505f023-1951-4713-b55f-11b4f056b35b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:12.394916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505f023-1951-4713-b55f-11b4f056b35b service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:12.405245 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 721/1461] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:10:14.492853 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a428d-b1fa-4939-b29e-897421f9c519 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:14.493428 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a428d-b1fa-4939-b29e-897421f9c519 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:14.493428 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a428d-b1fa-4939-b29e-897421f9c519 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:14.494802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 741/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:14.500335 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd1e19-a34f-4829-82a5-90eb2b0ec50f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:14.500867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd1e19-a34f-4829-82a5-90eb2b0ec50f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:14.500867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd1e19-a34f-4829-82a5-90eb2b0ec50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:14.510449 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=79 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:10:14.527580 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5edd1e19-a34f-4829-82a5-90eb2b0ec50f None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:14.528847 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5edd1e19-a34f-4829-82a5-90eb2b0ec50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:14.538102 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 722/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:10:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:14.577079 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a428d-b1fa-4939-b29e-897421f9c519 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:14.583240 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=73 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:10:14.590071 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a428d-b1fa-4939-b29e-897421f9c519 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:14.594068 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79afac-d798-462a-a19a-ab8c511e6fb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:14.594757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79afac-d798-462a-a19a-ab8c511e6fb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:14.595038 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79afac-d798-462a-a19a-ab8c511e6fb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:14.595309 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79afac-d798-462a-a19a-ab8c511e6fb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:14.604843 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79afac-d798-462a-a19a-ab8c511e6fb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:14.606297 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79afac-d798-462a-a19a-ab8c511e6fb3 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:14.615752 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 742/1464] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:10:16.685008 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8b18b-f9d0-4ebb-b009-b83b037c5663 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:16.685615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8b18b-f9d0-4ebb-b009-b83b037c5663 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:16.685615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8b18b-f9d0-4ebb-b009-b83b037c5663 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:16.686960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 723/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:16.693373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8b874-209e-4e31-83af-31be06511a0b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:16.693808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8b874-209e-4e31-83af-31be06511a0b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:16.693926 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8b874-209e-4e31-83af-31be06511a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:16.717983 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b8c8b874-209e-4e31-83af-31be06511a0b None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:16.719221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b8c8b874-209e-4e31-83af-31be06511a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:16.728420 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 743/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:16.765290 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8b18b-f9d0-4ebb-b009-b83b037c5663 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:16.778463 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8b18b-f9d0-4ebb-b009-b83b037c5663 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:16.782863 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc216d56-4fa8-47d6-8040-112fcac42d04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:16.783372 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc216d56-4fa8-47d6-8040-112fcac42d04 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:16.783659 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc216d56-4fa8-47d6-8040-112fcac42d04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:16.783942 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc216d56-4fa8-47d6-8040-112fcac42d04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:16.793977 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc216d56-4fa8-47d6-8040-112fcac42d04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:16.795419 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc216d56-4fa8-47d6-8040-112fcac42d04 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:16.805514 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 724/1467] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:10:18.829374 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add479b-34b8-4765-99f7-e147effc0303 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:18.830263 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add479b-34b8-4765-99f7-e147effc0303 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:18.830263 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add479b-34b8-4765-99f7-e147effc0303 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:18.832684 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 744/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:10:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:18.837999 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47432284-6647-4903-9959-61db3c5b04b5 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:18.838388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47432284-6647-4903-9959-61db3c5b04b5 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:18.838586 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47432284-6647-4903-9959-61db3c5b04b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:18.856983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-47432284-6647-4903-9959-61db3c5b04b5 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:18.858176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-47432284-6647-4903-9959-61db3c5b04b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:18.867337 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 725/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:10:18.907004 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add479b-34b8-4765-99f7-e147effc0303 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:18.919251 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add479b-34b8-4765-99f7-e147effc0303 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:18.923735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e2e05-bfa2-4a63-96eb-3313fcdeec1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:18.924404 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e2e05-bfa2-4a63-96eb-3313fcdeec1c service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:18.924681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e2e05-bfa2-4a63-96eb-3313fcdeec1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:18.924960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e2e05-bfa2-4a63-96eb-3313fcdeec1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:18.934331 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e2e05-bfa2-4a63-96eb-3313fcdeec1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:18.935812 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e2e05-bfa2-4a63-96eb-3313fcdeec1c service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:18.945187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 745/1470] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:10:28.851643 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=11 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:10:28.941002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=14 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:10:38.643320 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3e2f3-5eb3-4f17-b5e8-48a2a296b5b4 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:38.643973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3e2f3-5eb3-4f17-b5e8-48a2a296b5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:38.643973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3e2f3-5eb3-4f17-b5e8-48a2a296b5b4 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:38.645313 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 726/1471] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:38.652442 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b0784-7a12-4cd6-977f-f797955cda96 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:38.653194 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b0784-7a12-4cd6-977f-f797955cda96 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:38.653363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b0784-7a12-4cd6-977f-f797955cda96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:38.677477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2d1b0784-7a12-4cd6-977f-f797955cda96 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:38.678708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2d1b0784-7a12-4cd6-977f-f797955cda96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:38.687833 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 746/1472] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:38.737223 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dff320-3285-4409-a4eb-ae744d1cdce6 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:38.737513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dff320-3285-4409-a4eb-ae744d1cdce6 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:38.737789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dff320-3285-4409-a4eb-ae744d1cdce6 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:38.739185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 727/1473] 10.222.0.27 () {54 vars in 962 bytes} [Mon Jun 27 21:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:38.744206 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c86cb-4336-4fb5-a25b-7627b024aafa None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:38.744494 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c86cb-4336-4fb5-a25b-7627b024aafa None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:38.744765 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c86cb-4336-4fb5-a25b-7627b024aafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:38.763913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-be2c86cb-4336-4fb5-a25b-7627b024aafa None None] MFA Rules not processed for user `b43db1fd0b494a3f8588b498b585f247`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:38.765109 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-be2c86cb-4336-4fb5-a25b-7627b024aafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:38.772657 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 747/1474] 10.222.0.27 () {58 vars in 1059 bytes} [Mon Jun 27 21:10:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:38.780599 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dff320-3285-4409-a4eb-ae744d1cdce6 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:38.792964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dff320-3285-4409-a4eb-ae744d1cdce6 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:38.796805 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616533f-92ad-48d1-ae52-d17e3b01018c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b43db1fd0b494a3f8588b498b585f247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:38.797446 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616533f-92ad-48d1-ae52-d17e3b01018c service glance] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:38.797743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616533f-92ad-48d1-ae52-d17e3b01018c service glance] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:38.797983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616533f-92ad-48d1-ae52-d17e3b01018c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:38.807538 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616533f-92ad-48d1-ae52-d17e3b01018c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:38.809024 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616533f-92ad-48d1-ae52-d17e3b01018c service glance] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:38.818714 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 728/1475] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:10:40.958336 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80dc90f-dbe0-4f7d-b093-285e66d6d1f0 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:40.958895 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80dc90f-dbe0-4f7d-b093-285e66d6d1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:40.958895 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80dc90f-dbe0-4f7d-b093-285e66d6d1f0 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:40.960372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 748/1476] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:40.965747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fccc59-7a42-4265-9faf-f247c8e12c26 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:40.966172 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fccc59-7a42-4265-9faf-f247c8e12c26 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:40.966284 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fccc59-7a42-4265-9faf-f247c8e12c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:40.985093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-41fccc59-7a42-4265-9faf-f247c8e12c26 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:40.986312 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-41fccc59-7a42-4265-9faf-f247c8e12c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:40.996998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 729/1477] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:10:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:41.020833 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80dc90f-dbe0-4f7d-b093-285e66d6d1f0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:41.035462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80dc90f-dbe0-4f7d-b093-285e66d6d1f0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:41.040520 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752e212-4454-409e-8440-2312ff90db3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:41.041177 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752e212-4454-409e-8440-2312ff90db3a admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:41.041457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752e212-4454-409e-8440-2312ff90db3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:41.041782 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752e212-4454-409e-8440-2312ff90db3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:41.042466 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752e212-4454-409e-8440-2312ff90db3a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:41.044304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-8752e212-4454-409e-8440-2312ff90db3a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:10:41.044304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 27 21:10:41.044304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: Jun 27 21:10:41.045461 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752e212-4454-409e-8440-2312ff90db3a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:41.065099 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 749/1478] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Jun 27 21:10:41 2022] GET /identity/v3/projects => generated 2553 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:10:43.116922 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6418c-8dee-4bc1-bff2-8a2df74e3a85 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:43.117497 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6418c-8dee-4bc1-bff2-8a2df74e3a85 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:43.117497 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6418c-8dee-4bc1-bff2-8a2df74e3a85 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:43.118813 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 730/1479] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:43.125598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d977d-9709-4387-be28-fd5455899543 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:43.126339 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d977d-9709-4387-be28-fd5455899543 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:43.126502 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d977d-9709-4387-be28-fd5455899543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:43.148181 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f75d977d-9709-4387-be28-fd5455899543 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:43.149748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f75d977d-9709-4387-be28-fd5455899543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:43.158561 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 750/1480] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:43.202196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6418c-8dee-4bc1-bff2-8a2df74e3a85 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:43.218401 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6418c-8dee-4bc1-bff2-8a2df74e3a85 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:43.225218 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919aa3ab-36b9-4cb3-b3db-203fda7e1b47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:43.225861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919aa3ab-36b9-4cb3-b3db-203fda7e1b47 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:43.226147 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919aa3ab-36b9-4cb3-b3db-203fda7e1b47 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:43.226533 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919aa3ab-36b9-4cb3-b3db-203fda7e1b47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:43.235004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919aa3ab-36b9-4cb3-b3db-203fda7e1b47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:43.236629 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919aa3ab-36b9-4cb3-b3db-203fda7e1b47 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:43.246237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 731/1481] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:10:45.452925 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90af52-d7d3-4585-8237-61e834c0150f None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:45.453499 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90af52-d7d3-4585-8237-61e834c0150f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:45.453499 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90af52-d7d3-4585-8237-61e834c0150f None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:45.454883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 751/1482] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:45.461336 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4094d9-463c-4fb2-8bfb-c3a02690dddb None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:45.462027 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4094d9-463c-4fb2-8bfb-c3a02690dddb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:45.462171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4094d9-463c-4fb2-8bfb-c3a02690dddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:45.486609 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-bf4094d9-463c-4fb2-8bfb-c3a02690dddb None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:45.487842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-bf4094d9-463c-4fb2-8bfb-c3a02690dddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:45.497235 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 732/1483] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:10:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:45.536296 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90af52-d7d3-4585-8237-61e834c0150f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:45.548071 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90af52-d7d3-4585-8237-61e834c0150f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:45.552018 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249472f8-f0e3-45be-92cf-5259c30cc36b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:45.552655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249472f8-f0e3-45be-92cf-5259c30cc36b service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:45.552933 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249472f8-f0e3-45be-92cf-5259c30cc36b service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:45.553207 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249472f8-f0e3-45be-92cf-5259c30cc36b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:45.562484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249472f8-f0e3-45be-92cf-5259c30cc36b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:45.563964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249472f8-f0e3-45be-92cf-5259c30cc36b service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:45.573529 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 752/1484] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:10:47.630996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4402e4c-1d2f-4e5d-89a9-fa71e68511f1 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:47.631983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4402e4c-1d2f-4e5d-89a9-fa71e68511f1 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:47.631983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4402e4c-1d2f-4e5d-89a9-fa71e68511f1 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:47.633720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 733/1485] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:10:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:47.639277 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41443ccd-ef9b-42d8-a963-646047fea05e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:47.639813 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41443ccd-ef9b-42d8-a963-646047fea05e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:47.639926 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41443ccd-ef9b-42d8-a963-646047fea05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:47.663256 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-41443ccd-ef9b-42d8-a963-646047fea05e None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:47.664933 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-41443ccd-ef9b-42d8-a963-646047fea05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:47.675230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 753/1486] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:10:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:47.713791 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4402e4c-1d2f-4e5d-89a9-fa71e68511f1 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:47.725973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4402e4c-1d2f-4e5d-89a9-fa71e68511f1 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:47.730034 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d70ee-683f-4b25-9737-ce4f9335248b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:47.730687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d70ee-683f-4b25-9737-ce4f9335248b service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:47.730964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d70ee-683f-4b25-9737-ce4f9335248b service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:47.731244 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d70ee-683f-4b25-9737-ce4f9335248b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:47.740595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64d70ee-683f-4b25-9737-ce4f9335248b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:47.742070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64d70ee-683f-4b25-9737-ce4f9335248b service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:47.751653 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 734/1487] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:10:49.874368 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313db471-019f-4002-b182-63b74c4de23d None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:49.874987 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313db471-019f-4002-b182-63b74c4de23d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:49.874987 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313db471-019f-4002-b182-63b74c4de23d None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:49.876365 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 754/1488] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:49.882774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a052b-9637-4edd-ac27-a4e0807f8ff5 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:49.883249 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a052b-9637-4edd-ac27-a4e0807f8ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:49.883356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a052b-9637-4edd-ac27-a4e0807f8ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:49.904534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ef0a052b-9637-4edd-ac27-a4e0807f8ff5 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:49.905750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ef0a052b-9637-4edd-ac27-a4e0807f8ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:49.915947 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 735/1489] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:49.964871 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313db471-019f-4002-b182-63b74c4de23d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:49.982741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313db471-019f-4002-b182-63b74c4de23d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:49.987515 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491bc3d-3ab8-419e-ba94-d4690cdd3de6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:49.988303 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491bc3d-3ab8-419e-ba94-d4690cdd3de6 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:49.989703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491bc3d-3ab8-419e-ba94-d4690cdd3de6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:49.989703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491bc3d-3ab8-419e-ba94-d4690cdd3de6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:50.002988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f491bc3d-3ab8-419e-ba94-d4690cdd3de6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:50.005227 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f491bc3d-3ab8-419e-ba94-d4690cdd3de6 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:50.016862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 755/1490] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:10:52.220212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b638a-b424-436c-91d9-6fbb0f452e6b None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:52.221050 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b638a-b424-436c-91d9-6fbb0f452e6b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:52.221050 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b638a-b424-436c-91d9-6fbb0f452e6b None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:52.223241 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 736/1491] 10.222.0.27 () {54 vars in 926 bytes} [Mon Jun 27 21:10:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:52.228934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657d8b6-e1dc-4086-bda5-28537bc6185d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:52.229368 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657d8b6-e1dc-4086-bda5-28537bc6185d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:52.229469 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657d8b6-e1dc-4086-bda5-28537bc6185d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:52.248503 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9657d8b6-e1dc-4086-bda5-28537bc6185d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:52.249708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9657d8b6-e1dc-4086-bda5-28537bc6185d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:52.259102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 756/1492] 10.222.0.27 () {58 vars in 1023 bytes} [Mon Jun 27 21:10:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:52.300975 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b638a-b424-436c-91d9-6fbb0f452e6b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:52.313810 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b638a-b424-436c-91d9-6fbb0f452e6b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:52.318259 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f88d66-0c7e-4502-a97f-20480e696c33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:52.318895 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f88d66-0c7e-4502-a97f-20480e696c33 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:52.319171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f88d66-0c7e-4502-a97f-20480e696c33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:52.319442 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f88d66-0c7e-4502-a97f-20480e696c33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:52.329269 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f88d66-0c7e-4502-a97f-20480e696c33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:52.330706 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f88d66-0c7e-4502-a97f-20480e696c33 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:52.340435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 737/1493] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:11:02.243323 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=33 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:11:02.336107 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=33 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:12:16.362056 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c1a7b-27ff-4df1-b0fd-caa1df558bc2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:16.363028 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c1a7b-27ff-4df1-b0fd-caa1df558bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:16.363532 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c1a7b-27ff-4df1-b0fd-caa1df558bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:16.387533 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ad9c1a7b-27ff-4df1-b0fd-caa1df558bc2 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:12:16.388783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ad9c1a7b-27ff-4df1-b0fd-caa1df558bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:12:16.399093 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 757/1494] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:12:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:12:16.407983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce21654-3c50-434f-8476-191fe01ca81a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:16.408494 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce21654-3c50-434f-8476-191fe01ca81a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:16.408494 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce21654-3c50-434f-8476-191fe01ca81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:16.428656 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-0ce21654-3c50-434f-8476-191fe01ca81a None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:12:16.429868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0ce21654-3c50-434f-8476-191fe01ca81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:12:16.439249 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 738/1495] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:12:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:12:16.443841 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c1a7b-27ff-4df1-b0fd-caa1df558bc2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:12:16.454006 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c1a7b-27ff-4df1-b0fd-caa1df558bc2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:12:16.459274 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db42f04-1968-42e0-a75b-b65603b3832b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:12:16.459975 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db42f04-1968-42e0-a75b-b65603b3832b admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:16.460407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db42f04-1968-42e0-a75b-b65603b3832b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:16.460689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db42f04-1968-42e0-a75b-b65603b3832b admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:16.461404 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db42f04-1968-42e0-a75b-b65603b3832b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:12:16.461707 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db42f04-1968-42e0-a75b-b65603b3832b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:12:16.462786 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-6db42f04-1968-42e0-a75b-b65603b3832b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:12:16.462786 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 27 21:12:16.462786 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: Jun 27 21:12:16.463705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db42f04-1968-42e0-a75b-b65603b3832b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:12:16.475321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 758/1496] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:12:16.483245 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce21654-3c50-434f-8476-191fe01ca81a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:12:16.492944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce21654-3c50-434f-8476-191fe01ca81a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:12:16.499230 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa424a6-5b52-46e9-b2f7-c04945151e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:12:16.499920 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa424a6-5b52-46e9-b2f7-c04945151e2d admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:16.500203 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa424a6-5b52-46e9-b2f7-c04945151e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:16.500475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa424a6-5b52-46e9-b2f7-c04945151e2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:16.501175 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa424a6-5b52-46e9-b2f7-c04945151e2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:12:16.502989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: WARNING py.warnings [None req-9fa424a6-5b52-46e9-b2f7-c04945151e2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:12:16.502989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: warnings.warn(msg) Jun 27 21:12:16.502989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: Jun 27 21:12:16.504232 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa424a6-5b52-46e9-b2f7-c04945151e2d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:12:16.546183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 739/1497] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:12:16 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:12:16.552418 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db42f04-1968-42e0-a75b-b65603b3832b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:12:16.564408 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db42f04-1968-42e0-a75b-b65603b3832b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:12:16.623547 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca4102-6ee3-4b2a-8241-9d15f46db7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:12:16.624186 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca4102-6ee3-4b2a-8241-9d15f46db7f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:16.624474 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca4102-6ee3-4b2a-8241-9d15f46db7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:16.624755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca4102-6ee3-4b2a-8241-9d15f46db7f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:16.625442 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca4102-6ee3-4b2a-8241-9d15f46db7f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:12:16.626509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-41ca4102-6ee3-4b2a-8241-9d15f46db7f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:12:16.626509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 27 21:12:16.626509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: Jun 27 21:12:16.627755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca4102-6ee3-4b2a-8241-9d15f46db7f4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:12:16.653887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 759/1498] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:12:16 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:12:16.658145 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa424a6-5b52-46e9-b2f7-c04945151e2d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:12:16.669136 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa424a6-5b52-46e9-b2f7-c04945151e2d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:12:16.674493 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d367802-3bf4-4d34-96fe-0677a1324e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:12:16.675180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d367802-3bf4-4d34-96fe-0677a1324e32 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:16.675470 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d367802-3bf4-4d34-96fe-0677a1324e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:16.675783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d367802-3bf4-4d34-96fe-0677a1324e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:16.676518 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d367802-3bf4-4d34-96fe-0677a1324e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:12:16.677554 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: WARNING py.warnings [None req-2d367802-3bf4-4d34-96fe-0677a1324e32 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:12:16.677554 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: warnings.warn(msg) Jun 27 21:12:16.677554 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: Jun 27 21:12:16.678458 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d367802-3bf4-4d34-96fe-0677a1324e32 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:12:16.691948 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 740/1499] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:12:16.695966 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca4102-6ee3-4b2a-8241-9d15f46db7f4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:12:16.706321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca4102-6ee3-4b2a-8241-9d15f46db7f4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:12:16.711442 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d35ca-d64e-4071-9817-fa2c9853af90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:12:16.712138 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d35ca-d64e-4071-9817-fa2c9853af90 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:16.712428 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d35ca-d64e-4071-9817-fa2c9853af90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:16.712683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d35ca-d64e-4071-9817-fa2c9853af90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:16.713413 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d35ca-d64e-4071-9817-fa2c9853af90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:12:16.714436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-a06d35ca-d64e-4071-9817-fa2c9853af90 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:12:16.714436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 27 21:12:16.714436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: Jun 27 21:12:16.715330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d35ca-d64e-4071-9817-fa2c9853af90 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:12:16.729100 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 760/1500] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:12:16.733334 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d367802-3bf4-4d34-96fe-0677a1324e32 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:12:16.745260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d367802-3bf4-4d34-96fe-0677a1324e32 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:12:16.749792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13935ce-ffd3-4a12-ad56-7dc59ae542b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:12:16.750681 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13935ce-ffd3-4a12-ad56-7dc59ae542b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:16.750953 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13935ce-ffd3-4a12-ad56-7dc59ae542b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:16.751217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13935ce-ffd3-4a12-ad56-7dc59ae542b4 admin admin] PATH_INFO: `/v3/projects/95e28c053aba4b00b98cba0c6baeea4d/users/182083ae15194282980bdd07ddff3555/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:16.767279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13935ce-ffd3-4a12-ad56-7dc59ae542b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95e28c053aba4b00b98cba0c6baeea4d, user_id=182083ae15194282980bdd07ddff3555, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:12:16.768368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: WARNING py.warnings [None req-d13935ce-ffd3-4a12-ad56-7dc59ae542b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:12:16.768368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: warnings.warn(msg) Jun 27 21:12:16.768368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: Jun 27 21:12:16.770340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13935ce-ffd3-4a12-ad56-7dc59ae542b4 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:12:16.784463 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 741/1501] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:12:16 2022] PUT /identity/v3/projects/95e28c053aba4b00b98cba0c6baeea4d/users/182083ae15194282980bdd07ddff3555/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:12:16.795988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf81ae-a34a-4bfd-8118-cd2bfbba4660 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:16.796277 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf81ae-a34a-4bfd-8118-cd2bfbba4660 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:16.796547 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf81ae-a34a-4bfd-8118-cd2bfbba4660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:16.824260 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a1bf81ae-a34a-4bfd-8118-cd2bfbba4660 None None] MFA Rules not processed for user `182083ae15194282980bdd07ddff3555`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:12:16.825555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a1bf81ae-a34a-4bfd-8118-cd2bfbba4660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:12:16.891629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 761/1502] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:12:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:12:16.905674 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443a2fa-9ed2-49c8-8347-25b649f8faad None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:16.905980 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443a2fa-9ed2-49c8-8347-25b649f8faad None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:16.906255 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443a2fa-9ed2-49c8-8347-25b649f8faad None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:16.907641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 742/1503] 10.222.0.27 () {54 vars in 963 bytes} [Mon Jun 27 21:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:12:16.912692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db36fb0-53c1-455b-b450-1840dcf22f18 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:16.912983 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db36fb0-53c1-455b-b450-1840dcf22f18 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:16.913253 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db36fb0-53c1-455b-b450-1840dcf22f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:16.932804 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9db36fb0-53c1-455b-b450-1840dcf22f18 None None] MFA Rules not processed for user `16fda36445d9415aae4df74f31af0601`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:12:16.934009 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9db36fb0-53c1-455b-b450-1840dcf22f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:12:16.980113 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:12:16.994942 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 762/1504] 10.222.0.27 () {58 vars in 1060 bytes} [Mon Jun 27 21:12:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:12:17.002086 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443a2fa-9ed2-49c8-8347-25b649f8faad None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:12:17.019790 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443a2fa-9ed2-49c8-8347-25b649f8faad None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:12:17.025535 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929ea36-2ebe-470b-9b90-0ef0ab61e17b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:12:17.026180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929ea36-2ebe-470b-9b90-0ef0ab61e17b service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:17.026453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929ea36-2ebe-470b-9b90-0ef0ab61e17b service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:17.026725 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929ea36-2ebe-470b-9b90-0ef0ab61e17b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:17.037406 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929ea36-2ebe-470b-9b90-0ef0ab61e17b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:12:17.038848 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929ea36-2ebe-470b-9b90-0ef0ab61e17b service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:12:17.048377 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 743/1505] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:12:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:12:17.090209 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67653ed-af7d-433b-84b2-9057a58eef14 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:17.090495 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67653ed-af7d-433b-84b2-9057a58eef14 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:17.090798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67653ed-af7d-433b-84b2-9057a58eef14 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:17.092167 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 763/1506] 10.222.0.27 () {52 vars in 829 bytes} [Mon Jun 27 21:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:12:17.096054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929ea36-2ebe-470b-9b90-0ef0ab61e17b service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:12:17.162585 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929ea36-2ebe-470b-9b90-0ef0ab61e17b service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:12:17.167520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8c2b2-c801-4fa3-9ced-c19b42e13a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:12:17.168439 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8c2b2-c801-4fa3-9ced-c19b42e13a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:17.168720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8c2b2-c801-4fa3-9ced-c19b42e13a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:17.168995 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8c2b2-c801-4fa3-9ced-c19b42e13a4e admin admin] PATH_INFO: `/v3/users/182083ae15194282980bdd07ddff3555` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:17.170226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8c2b2-c801-4fa3-9ced-c19b42e13a4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=182083ae15194282980bdd07ddff3555)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:12:17.171262 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: WARNING py.warnings [None req-c3b8c2b2-c801-4fa3-9ced-c19b42e13a4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:12:17.171262 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: warnings.warn(msg) Jun 27 21:12:17.171262 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: Jun 27 21:12:17.172522 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8c2b2-c801-4fa3-9ced-c19b42e13a4e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:12:17.236994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c3b8c2b2-c801-4fa3-9ced-c19b42e13a4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '182083ae15194282980bdd07ddff3555'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:12:17.237739 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c3b8c2b2-c801-4fa3-9ced-c19b42e13a4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '182083ae15194282980bdd07ddff3555'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:12:17.238387 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c3b8c2b2-c801-4fa3-9ced-c19b42e13a4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '182083ae15194282980bdd07ddff3555'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:12:17.257243 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c3b8c2b2-c801-4fa3-9ced-c19b42e13a4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '182083ae15194282980bdd07ddff3555'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:12:17.269985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c3b8c2b2-c801-4fa3-9ced-c19b42e13a4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '182083ae15194282980bdd07ddff3555'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:12:17.274559 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:12:17.287508 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 744/1507] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:12:17 2022] DELETE /identity/v3/users/182083ae15194282980bdd07ddff3555 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:12:17.292181 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fe8c8-d58e-43d9-a3d6-2ae4a24e928d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:17.292469 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fe8c8-d58e-43d9-a3d6-2ae4a24e928d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:17.292748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fe8c8-d58e-43d9-a3d6-2ae4a24e928d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:17.310850 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-913fe8c8-d58e-43d9-a3d6-2ae4a24e928d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:12:17.312133 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-913fe8c8-d58e-43d9-a3d6-2ae4a24e928d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:12:17.377859 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 764/1508] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:12:17.390989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8c2b2-c801-4fa3-9ced-c19b42e13a4e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:12:17.393179 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c3b8c2b2-c801-4fa3-9ced-c19b42e13a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:12:17.452989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8c2b2-c801-4fa3-9ced-c19b42e13a4e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:12:17.457818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76b6cd-322e-44c4-a62d-ae43927fe0e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:12:17.458481 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76b6cd-322e-44c4-a62d-ae43927fe0e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:17.458789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76b6cd-322e-44c4-a62d-ae43927fe0e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:17.459068 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76b6cd-322e-44c4-a62d-ae43927fe0e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:17.469572 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76b6cd-322e-44c4-a62d-ae43927fe0e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:12:17.471079 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76b6cd-322e-44c4-a62d-ae43927fe0e6 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:12:17.480839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 745/1509] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:12:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:12:17.813191 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fe8c8-d58e-43d9-a3d6-2ae4a24e928d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:12:17.814813 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-913fe8c8-d58e-43d9-a3d6-2ae4a24e928d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:12:17.827036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fe8c8-d58e-43d9-a3d6-2ae4a24e928d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:12:17.832487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96a540-610b-4abb-9e12-92dbb5f3073b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:12:17.833381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96a540-610b-4abb-9e12-92dbb5f3073b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:17.833670 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96a540-610b-4abb-9e12-92dbb5f3073b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:17.833939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96a540-610b-4abb-9e12-92dbb5f3073b admin admin] PATH_INFO: `/v3/projects/95e28c053aba4b00b98cba0c6baeea4d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:17.835155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96a540-610b-4abb-9e12-92dbb5f3073b admin admin] RBAC: Authorizing `identity:delete_project(project_id=95e28c053aba4b00b98cba0c6baeea4d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:12:17.836248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-0c96a540-610b-4abb-9e12-92dbb5f3073b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:12:17.836248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 27 21:12:17.836248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: Jun 27 21:12:17.837448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96a540-610b-4abb-9e12-92dbb5f3073b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:12:17.901761 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0c96a540-610b-4abb-9e12-92dbb5f3073b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95e28c053aba4b00b98cba0c6baeea4d', 'request_id': 'req-0c96a540-610b-4abb-9e12-92dbb5f3073b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:12:17.911909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0c96a540-610b-4abb-9e12-92dbb5f3073b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95e28c053aba4b00b98cba0c6baeea4d', 'request_id': 'req-0c96a540-610b-4abb-9e12-92dbb5f3073b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:12:17.918322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0c96a540-610b-4abb-9e12-92dbb5f3073b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95e28c053aba4b00b98cba0c6baeea4d', 'request_id': 'req-0c96a540-610b-4abb-9e12-92dbb5f3073b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:12:17.936911 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0c96a540-610b-4abb-9e12-92dbb5f3073b admin admin] The token cache is being invalidate because project 95e28c053aba4b00b98cba0c6baeea4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:12:17.937198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0c96a540-610b-4abb-9e12-92dbb5f3073b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:12:17.937888 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0c96a540-610b-4abb-9e12-92dbb5f3073b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:12:17.939655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 765/1510] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:12:17 2022] DELETE /identity/v3/projects/95e28c053aba4b00b98cba0c6baeea4d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:12:27.476475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:12:27.933945 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:13:53.851201 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ac35f-069c-4cbc-a675-4ae0b12c1e23 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:53.852214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ac35f-069c-4cbc-a675-4ae0b12c1e23 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:53.852214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ac35f-069c-4cbc-a675-4ae0b12c1e23 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:53.854324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 746/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:13:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:13:53.861489 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c0acd-d03c-4aa6-8513-6cda2ec38b88 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:53.861955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c0acd-d03c-4aa6-8513-6cda2ec38b88 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:53.862049 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c0acd-d03c-4aa6-8513-6cda2ec38b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:53.881770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-501c0acd-d03c-4aa6-8513-6cda2ec38b88 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:13:53.882963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-501c0acd-d03c-4aa6-8513-6cda2ec38b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:13:53.957579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 766/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:13:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:13:54.115842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ac35f-069c-4cbc-a675-4ae0b12c1e23 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:13:54.118764 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-159ac35f-069c-4cbc-a675-4ae0b12c1e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:13:54.206409 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ac35f-069c-4cbc-a675-4ae0b12c1e23 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:13:54.212820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be518ac7-c717-47cd-94c7-7c93683f5e5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:13:54.213477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be518ac7-c717-47cd-94c7-7c93683f5e5e service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:54.213766 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be518ac7-c717-47cd-94c7-7c93683f5e5e service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:54.214049 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be518ac7-c717-47cd-94c7-7c93683f5e5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:54.222585 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be518ac7-c717-47cd-94c7-7c93683f5e5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:13:54.224239 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be518ac7-c717-47cd-94c7-7c93683f5e5e service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:13:54.234181 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 747/1513] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:13:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:13:55.038278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8870aa-7b6c-4557-8e65-6855037d2c3a None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:55.038804 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8870aa-7b6c-4557-8e65-6855037d2c3a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:55.038804 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8870aa-7b6c-4557-8e65-6855037d2c3a None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:55.040316 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 767/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:13:55.045362 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1471ae-bffb-4913-a504-566bd9e47c21 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:55.045826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1471ae-bffb-4913-a504-566bd9e47c21 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:55.045934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1471ae-bffb-4913-a504-566bd9e47c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:55.065313 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-7e1471ae-bffb-4913-a504-566bd9e47c21 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:13:55.066510 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7e1471ae-bffb-4913-a504-566bd9e47c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:13:55.075307 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 748/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:13:55.230826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8870aa-7b6c-4557-8e65-6855037d2c3a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:13:55.251447 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8870aa-7b6c-4557-8e65-6855037d2c3a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:13:55.258236 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ac767-7dcf-47c8-a701-60ba2433ac61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:13:55.258876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ac767-7dcf-47c8-a701-60ba2433ac61 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:55.259196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ac767-7dcf-47c8-a701-60ba2433ac61 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:55.259466 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ac767-7dcf-47c8-a701-60ba2433ac61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:55.267860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ac767-7dcf-47c8-a701-60ba2433ac61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:13:55.269485 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ac767-7dcf-47c8-a701-60ba2433ac61 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:13:55.279337 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 768/1516] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:13:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:13:55.833521 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918338c-53cc-4cf9-8eb5-0a471b2d1aaf None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:55.833803 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918338c-53cc-4cf9-8eb5-0a471b2d1aaf None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:55.834111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918338c-53cc-4cf9-8eb5-0a471b2d1aaf None None] PATH_INFO: `/v3` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:55.835495 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 749/1517] 10.222.0.46 () {54 vars in 925 bytes} [Mon Jun 27 21:13:55 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:13:55.842665 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625ef98-4181-4952-979a-801514409366 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:55.842961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625ef98-4181-4952-979a-801514409366 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:55.843237 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625ef98-4181-4952-979a-801514409366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:55.862904 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b625ef98-4181-4952-979a-801514409366 None None] MFA Rules not processed for user `1f5fa9b0b0314ee9bb7e16b8464307b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:13:55.864158 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b625ef98-4181-4952-979a-801514409366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:13:55.932567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 769/1518] 10.222.0.46 () {58 vars in 1015 bytes} [Mon Jun 27 21:13:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:14:05.059643 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=37 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:14:05.471767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84758c91-e4ff-47e5-9ac1-b8fa5d6981b5 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:05.472014 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84758c91-e4ff-47e5-9ac1-b8fa5d6981b5 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:05.472221 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84758c91-e4ff-47e5-9ac1-b8fa5d6981b5 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:05.474105 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 750/1519] 10.222.0.27 () {54 vars in 960 bytes} [Mon Jun 27 21:14:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:14:05.480975 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbeba6c-1d21-43f1-8b46-00f643c042e8 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:05.482914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbeba6c-1d21-43f1-8b46-00f643c042e8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:05.482914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbeba6c-1d21-43f1-8b46-00f643c042e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:05.500894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3bbeba6c-1d21-43f1-8b46-00f643c042e8 None None] MFA Rules not processed for user `1f5fa9b0b0314ee9bb7e16b8464307b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:14:05.502116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3bbeba6c-1d21-43f1-8b46-00f643c042e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:14:05.511058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 770/1520] 10.222.0.27 () {58 vars in 1057 bytes} [Mon Jun 27 21:14:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:14:05.519273 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84758c91-e4ff-47e5-9ac1-b8fa5d6981b5 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:14:05.534554 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84758c91-e4ff-47e5-9ac1-b8fa5d6981b5 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:14:05.540252 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd08022-9427-4a7b-a891-0ccdc58d9144 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5fa9b0b0314ee9bb7e16b8464307b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:14:05.540909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd08022-9427-4a7b-a891-0ccdc58d9144 service placement] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:05.541196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd08022-9427-4a7b-a891-0ccdc58d9144 service placement] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:05.541478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd08022-9427-4a7b-a891-0ccdc58d9144 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:05.550725 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd08022-9427-4a7b-a891-0ccdc58d9144 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:14:05.552401 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd08022-9427-4a7b-a891-0ccdc58d9144 service placement] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:14:05.563228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 751/1521] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:14:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:14:06.536243 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d5922-97ed-4810-8cfc-fbac14f250e1 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:06.537048 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d5922-97ed-4810-8cfc-fbac14f250e1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:06.537048 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d5922-97ed-4810-8cfc-fbac14f250e1 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:06.539202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 771/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:14:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:14:06.547074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebbaaa-d938-4a05-a8c2-ba180c20c887 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:06.547824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebbaaa-d938-4a05-a8c2-ba180c20c887 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:06.548002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebbaaa-d938-4a05-a8c2-ba180c20c887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:06.575280 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-19ebbaaa-d938-4a05-a8c2-ba180c20c887 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:14:06.576530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-19ebbaaa-d938-4a05-a8c2-ba180c20c887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:14:06.585452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 752/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:14:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:14:06.737949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d5922-97ed-4810-8cfc-fbac14f250e1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:14:06.756204 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d5922-97ed-4810-8cfc-fbac14f250e1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:14:06.762721 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05025035-ddc3-44c6-b610-df9047cf1b4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:14:06.763370 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05025035-ddc3-44c6-b610-df9047cf1b4b service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:06.763688 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05025035-ddc3-44c6-b610-df9047cf1b4b service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:06.763971 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05025035-ddc3-44c6-b610-df9047cf1b4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:06.774134 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05025035-ddc3-44c6-b610-df9047cf1b4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:14:06.775989 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05025035-ddc3-44c6-b610-df9047cf1b4b service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:14:06.786565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 772/1524] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:14:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:14:07.591558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7791c5-b3a0-4b5b-90e1-54e725fd0ca3 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:07.592164 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7791c5-b3a0-4b5b-90e1-54e725fd0ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:07.592164 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7791c5-b3a0-4b5b-90e1-54e725fd0ca3 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:07.594423 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 753/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:14:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:14:07.602558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae13db-1fb8-4b0d-a035-f17451752316 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:07.603302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae13db-1fb8-4b0d-a035-f17451752316 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:07.603476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae13db-1fb8-4b0d-a035-f17451752316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:07.623753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7cae13db-1fb8-4b0d-a035-f17451752316 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:14:07.624987 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7cae13db-1fb8-4b0d-a035-f17451752316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:14:07.633823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 773/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:14:07.794714 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7791c5-b3a0-4b5b-90e1-54e725fd0ca3 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:14:07.818646 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7791c5-b3a0-4b5b-90e1-54e725fd0ca3 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:14:07.826556 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e116dd1d-06e3-4a0f-8b8b-1aa699178fdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:14:07.827207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e116dd1d-06e3-4a0f-8b8b-1aa699178fdf service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:07.827489 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e116dd1d-06e3-4a0f-8b8b-1aa699178fdf service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:07.827802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e116dd1d-06e3-4a0f-8b8b-1aa699178fdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:07.836706 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e116dd1d-06e3-4a0f-8b8b-1aa699178fdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:14:07.838323 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e116dd1d-06e3-4a0f-8b8b-1aa699178fdf service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:14:07.848486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 754/1527] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:14:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:14:11.755821 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2dc4d6-c94e-451d-8b8f-d0c94638b522 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:11.756823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2dc4d6-c94e-451d-8b8f-d0c94638b522 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:11.756823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2dc4d6-c94e-451d-8b8f-d0c94638b522 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:11.758905 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 774/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:14:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:14:11.764559 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de33b4-3883-4f3c-9989-6cd0a28fc5b1 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:11.765018 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de33b4-3883-4f3c-9989-6cd0a28fc5b1 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:11.765124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de33b4-3883-4f3c-9989-6cd0a28fc5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:11.787903 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-40de33b4-3883-4f3c-9989-6cd0a28fc5b1 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:14:11.789160 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-40de33b4-3883-4f3c-9989-6cd0a28fc5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:14:11.798694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 755/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:14:11.836567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2dc4d6-c94e-451d-8b8f-d0c94638b522 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:14:11.838593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3d2dc4d6-c94e-451d-8b8f-d0c94638b522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:14:11.911620 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2dc4d6-c94e-451d-8b8f-d0c94638b522 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:14:11.915897 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500ffd2-c571-4b93-b030-6fe8edd14f8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:14:11.916589 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500ffd2-c571-4b93-b030-6fe8edd14f8f service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:11.916872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500ffd2-c571-4b93-b030-6fe8edd14f8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:11.917228 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500ffd2-c571-4b93-b030-6fe8edd14f8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:11.926512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500ffd2-c571-4b93-b030-6fe8edd14f8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:14:11.928021 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500ffd2-c571-4b93-b030-6fe8edd14f8f service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:14:11.933098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:14:11.938711 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 775/1530] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:14:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:14:13.923383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c675ca9-0079-4cf8-a6c1-77d46ed8be02 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:13.924200 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c675ca9-0079-4cf8-a6c1-77d46ed8be02 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:13.924373 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c675ca9-0079-4cf8-a6c1-77d46ed8be02 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:13.926552 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 756/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:14:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:14:13.931863 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c7fc1-55d9-4ca2-aea1-59949aa48f4b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:13.932335 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c7fc1-55d9-4ca2-aea1-59949aa48f4b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:13.932451 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c7fc1-55d9-4ca2-aea1-59949aa48f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:13.951225 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9a1c7fc1-55d9-4ca2-aea1-59949aa48f4b None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:14:13.952517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9a1c7fc1-55d9-4ca2-aea1-59949aa48f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:14:13.961329 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 776/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:14:13.996709 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c675ca9-0079-4cf8-a6c1-77d46ed8be02 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:14:14.008962 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c675ca9-0079-4cf8-a6c1-77d46ed8be02 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:14:14.013028 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e50b82-fdcb-43f4-8083-885f10261212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:14:14.013671 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e50b82-fdcb-43f4-8083-885f10261212 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:14.013956 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e50b82-fdcb-43f4-8083-885f10261212 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:14.014682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e50b82-fdcb-43f4-8083-885f10261212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:14.025064 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e50b82-fdcb-43f4-8083-885f10261212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:14:14.026505 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e50b82-fdcb-43f4-8083-885f10261212 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:14:14.036096 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 757/1533] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:14:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:14:17.346327 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadb886-8356-4b36-973f-8823b20236a0 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:17.346964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadb886-8356-4b36-973f-8823b20236a0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:17.346964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadb886-8356-4b36-973f-8823b20236a0 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:17.348314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 777/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:14:17.356910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13142338-2c3e-4c5d-b6a4-3f2c7515c31d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:17.357666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13142338-2c3e-4c5d-b6a4-3f2c7515c31d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:17.357853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13142338-2c3e-4c5d-b6a4-3f2c7515c31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:17.377019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-13142338-2c3e-4c5d-b6a4-3f2c7515c31d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:14:17.378212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-13142338-2c3e-4c5d-b6a4-3f2c7515c31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:14:17.386636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 758/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:14:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:14:17.693834 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eadb886-8356-4b36-973f-8823b20236a0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:14:17.696564 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5eadb886-8356-4b36-973f-8823b20236a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:14:17.764631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eadb886-8356-4b36-973f-8823b20236a0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:14:17.768631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfe3e9-a9a6-4c35-9e81-fb8ce4a79703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:14:17.769269 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfe3e9-a9a6-4c35-9e81-fb8ce4a79703 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:17.769550 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfe3e9-a9a6-4c35-9e81-fb8ce4a79703 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:17.769826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfe3e9-a9a6-4c35-9e81-fb8ce4a79703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:17.779597 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfe3e9-a9a6-4c35-9e81-fb8ce4a79703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:14:17.781061 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfe3e9-a9a6-4c35-9e81-fb8ce4a79703 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:14:17.790654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 778/1536] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:14:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:14:19.858391 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a618b-91d7-4bec-950d-8a5186ed763c None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:19.858996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a618b-91d7-4bec-950d-8a5186ed763c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:19.858996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a618b-91d7-4bec-950d-8a5186ed763c None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:19.860383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 759/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:14:19.866460 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff725600-b9f7-4523-be1d-3858f51a46e8 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:19.866919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff725600-b9f7-4523-be1d-3858f51a46e8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:19.867034 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff725600-b9f7-4523-be1d-3858f51a46e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:19.888930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ff725600-b9f7-4523-be1d-3858f51a46e8 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:14:19.890130 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ff725600-b9f7-4523-be1d-3858f51a46e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:14:19.899225 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 779/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:14:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:14:20.204043 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a618b-91d7-4bec-950d-8a5186ed763c None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:14:20.205887 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a93a618b-91d7-4bec-950d-8a5186ed763c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:14:20.219973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a618b-91d7-4bec-950d-8a5186ed763c None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:14:20.224608 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328d870-211b-41ca-9b56-4acdb0e73cb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:14:20.225277 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328d870-211b-41ca-9b56-4acdb0e73cb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:20.225564 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328d870-211b-41ca-9b56-4acdb0e73cb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:20.225859 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328d870-211b-41ca-9b56-4acdb0e73cb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:20.235908 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328d870-211b-41ca-9b56-4acdb0e73cb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:14:20.237362 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328d870-211b-41ca-9b56-4acdb0e73cb3 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:14:20.246985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 760/1539] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:14:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:14:29.883280 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=32 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:14:30.243227 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=33 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:48.274776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad88ba-fe7b-418e-b094-492ab7d080be None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.276030 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad88ba-fe7b-418e-b094-492ab7d080be None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.276568 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad88ba-fe7b-418e-b094-492ab7d080be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.302525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-73ad88ba-fe7b-418e-b094-492ab7d080be None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:48.304198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-73ad88ba-fe7b-418e-b094-492ab7d080be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:48.316014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 780/1540] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:48.326170 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9d813-cfb3-4fc7-bffa-0094030fc3ae None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.327238 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9d813-cfb3-4fc7-bffa-0094030fc3ae None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.327790 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9d813-cfb3-4fc7-bffa-0094030fc3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.361120 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-f1a9d813-cfb3-4fc7-bffa-0094030fc3ae None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:48.362361 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f1a9d813-cfb3-4fc7-bffa-0094030fc3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:48.374335 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 761/1541] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:48.375692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad88ba-fe7b-418e-b094-492ab7d080be None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:48.391638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad88ba-fe7b-418e-b094-492ab7d080be None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:48.405479 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015d9fc-777e-41d0-bb21-003e93edcf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:48.406925 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015d9fc-777e-41d0-bb21-003e93edcf09 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.407503 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015d9fc-777e-41d0-bb21-003e93edcf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.408031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015d9fc-777e-41d0-bb21-003e93edcf09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.409032 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015d9fc-777e-41d0-bb21-003e93edcf09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:15:48.409577 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015d9fc-777e-41d0-bb21-003e93edcf09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:48.411499 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015d9fc-777e-41d0-bb21-003e93edcf09 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:48.432698 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 781/1542] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:15:48.442117 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9d813-cfb3-4fc7-bffa-0094030fc3ae None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:48.456472 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9d813-cfb3-4fc7-bffa-0094030fc3ae None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:48.461589 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769d54f-b1eb-435e-8a85-926324ff90bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:48.462293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769d54f-b1eb-435e-8a85-926324ff90bd admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.462576 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769d54f-b1eb-435e-8a85-926324ff90bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.462854 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769d54f-b1eb-435e-8a85-926324ff90bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.463583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769d54f-b1eb-435e-8a85-926324ff90bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:48.465708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769d54f-b1eb-435e-8a85-926324ff90bd admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:48.501512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9ec8e-e4b6-492d-9ba7-f964fefe7a18 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.502521 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9ec8e-e4b6-492d-9ba7-f964fefe7a18 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.503099 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9ec8e-e4b6-492d-9ba7-f964fefe7a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.519981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 762/1543] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:48 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:48.527552 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769d54f-b1eb-435e-8a85-926324ff90bd admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:48.545280 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5da9ec8e-e4b6-492d-9ba7-f964fefe7a18 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:48.546958 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5da9ec8e-e4b6-492d-9ba7-f964fefe7a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:48.636568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769d54f-b1eb-435e-8a85-926324ff90bd admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:48.647423 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b5466-191c-4a5b-81bb-db1495b09fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:48.648139 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b5466-191c-4a5b-81bb-db1495b09fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.648421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b5466-191c-4a5b-81bb-db1495b09fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.648694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b5466-191c-4a5b-81bb-db1495b09fd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.649392 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b5466-191c-4a5b-81bb-db1495b09fd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:48.651017 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: WARNING py.warnings [None req-822b5466-191c-4a5b-81bb-db1495b09fd8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:15:48.651017 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: warnings.warn(msg) Jun 27 21:15:48.651017 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: Jun 27 21:15:48.652260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b5466-191c-4a5b-81bb-db1495b09fd8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:48.666246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 782/1544] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:15:48.667692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe2dfb-9300-4e28-870a-a88889eed301 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.667984 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe2dfb-9300-4e28-870a-a88889eed301 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.668254 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe2dfb-9300-4e28-870a-a88889eed301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.705981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-dabe2dfb-9300-4e28-870a-a88889eed301 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:48.707829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-dabe2dfb-9300-4e28-870a-a88889eed301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:48.715071 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 763/1545] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:48 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:15:48.715833 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c40c5b-182d-4b30-9615-663b66d4c970 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.716127 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c40c5b-182d-4b30-9615-663b66d4c970 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.716403 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c40c5b-182d-4b30-9615-663b66d4c970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.727450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 783/1546] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:48.728679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe2dfb-9300-4e28-870a-a88889eed301 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:48.747242 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe2dfb-9300-4e28-870a-a88889eed301 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:48.758479 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-45c40c5b-182d-4b30-9615-663b66d4c970 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:48.759724 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-45c40c5b-182d-4b30-9615-663b66d4c970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:48.763032 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300d133-a659-46fb-9d36-d08775dc1aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:48.763763 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300d133-a659-46fb-9d36-d08775dc1aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.764039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300d133-a659-46fb-9d36-d08775dc1aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.764315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300d133-a659-46fb-9d36-d08775dc1aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.765053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4300d133-a659-46fb-9d36-d08775dc1aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:48.767381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4300d133-a659-46fb-9d36-d08775dc1aa6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:48.776996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 764/1547] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:48.781844 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9520eb-498f-4686-9a0f-cb54d84bfb98 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.782109 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9520eb-498f-4686-9a0f-cb54d84bfb98 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.782396 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9520eb-498f-4686-9a0f-cb54d84bfb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.783871 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 784/1548] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:48.785354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba5c2c-47f0-4b13-b8dc-8191ea538126 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.785670 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba5c2c-47f0-4b13-b8dc-8191ea538126 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.785950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba5c2c-47f0-4b13-b8dc-8191ea538126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.802270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-4a9520eb-498f-4686-9a0f-cb54d84bfb98 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:48.803492 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4a9520eb-498f-4686-9a0f-cb54d84bfb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:48.807442 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-91ba5c2c-47f0-4b13-b8dc-8191ea538126 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:48.808639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-91ba5c2c-47f0-4b13-b8dc-8191ea538126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:48.811676 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 765/1549] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:48.812903 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33007ffa-ba9e-4e9e-b29f-6e8048570397 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.813189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33007ffa-ba9e-4e9e-b29f-6e8048570397 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.813506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33007ffa-ba9e-4e9e-b29f-6e8048570397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.819671 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 785/1550] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:48.819671 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc980d96-768a-47f2-bb87-e6a91b804e8f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.819671 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc980d96-768a-47f2-bb87-e6a91b804e8f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.819671 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc980d96-768a-47f2-bb87-e6a91b804e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.835119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-33007ffa-ba9e-4e9e-b29f-6e8048570397 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:48.836350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-33007ffa-ba9e-4e9e-b29f-6e8048570397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:48.838971 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-cc980d96-768a-47f2-bb87-e6a91b804e8f None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:48.840198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cc980d96-768a-47f2-bb87-e6a91b804e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:48.844519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 766/1551] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:48.845842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92109a6e-e53d-4b51-880a-065acb748abe None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.846132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92109a6e-e53d-4b51-880a-065acb748abe None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.846408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92109a6e-e53d-4b51-880a-065acb748abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.850407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 786/1552] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:48.851182 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc980d96-768a-47f2-bb87-e6a91b804e8f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:48.869230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc980d96-768a-47f2-bb87-e6a91b804e8f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:48.873425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-92109a6e-e53d-4b51-880a-065acb748abe None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:48.873853 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff12847-fb01-489f-b2d5-09b90dc77220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:48.874534 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff12847-fb01-489f-b2d5-09b90dc77220 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.874657 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-92109a6e-e53d-4b51-880a-065acb748abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:48.874816 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff12847-fb01-489f-b2d5-09b90dc77220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.875090 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff12847-fb01-489f-b2d5-09b90dc77220 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.875883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff12847-fb01-489f-b2d5-09b90dc77220 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:15:48.876194 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff12847-fb01-489f-b2d5-09b90dc77220 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:48.877829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff12847-fb01-489f-b2d5-09b90dc77220 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:48.882461 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 767/1553] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:48.883788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ba1a5-a4fa-4909-a377-e9a9c371d6be None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.884075 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ba1a5-a4fa-4909-a377-e9a9c371d6be None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.884351 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ba1a5-a4fa-4909-a377-e9a9c371d6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.895762 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 787/1554] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:15:48.896603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff12847-fb01-489f-b2d5-09b90dc77220 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:48.906097 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff12847-fb01-489f-b2d5-09b90dc77220 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:48.908378 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-c45ba1a5-a4fa-4909-a377-e9a9c371d6be None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:48.909587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c45ba1a5-a4fa-4909-a377-e9a9c371d6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:48.910599 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19062dbd-aea4-46ec-aa7e-7c8a1a9710ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:48.911251 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19062dbd-aea4-46ec-aa7e-7c8a1a9710ca admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.911533 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19062dbd-aea4-46ec-aa7e-7c8a1a9710ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.911840 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19062dbd-aea4-46ec-aa7e-7c8a1a9710ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.912558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19062dbd-aea4-46ec-aa7e-7c8a1a9710ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:48.914189 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19062dbd-aea4-46ec-aa7e-7c8a1a9710ca admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:48.917524 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 768/1555] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:48.918770 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec85c80-5f28-424a-a491-cc8bcdb36485 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.919054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec85c80-5f28-424a-a491-cc8bcdb36485 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.919330 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec85c80-5f28-424a-a491-cc8bcdb36485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.932961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 788/1556] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:48.932961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e30361-6779-48c1-9248-4601ffa5aba4 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.932961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e30361-6779-48c1-9248-4601ffa5aba4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.932961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e30361-6779-48c1-9248-4601ffa5aba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.941442 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-9ec85c80-5f28-424a-a491-cc8bcdb36485 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:48.942936 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9ec85c80-5f28-424a-a491-cc8bcdb36485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:48.949080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-90e30361-6779-48c1-9248-4601ffa5aba4 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:48.950245 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-90e30361-6779-48c1-9248-4601ffa5aba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:48.950913 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 769/1557] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:48.951729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec85c80-5f28-424a-a491-cc8bcdb36485 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:48.958166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 789/1558] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:48.959389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7e31c-4f81-471b-a830-c2dbd47ef427 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.959700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7e31c-4f81-471b-a830-c2dbd47ef427 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.959973 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7e31c-4f81-471b-a830-c2dbd47ef427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.961182 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec85c80-5f28-424a-a491-cc8bcdb36485 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:48.965899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10860963-730b-4b2d-ba30-b0a58f7ad889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:48.966746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10860963-730b-4b2d-ba30-b0a58f7ad889 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.967859 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10860963-730b-4b2d-ba30-b0a58f7ad889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.968155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10860963-730b-4b2d-ba30-b0a58f7ad889 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.968898 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10860963-730b-4b2d-ba30-b0a58f7ad889 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:15:48.969208 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10860963-730b-4b2d-ba30-b0a58f7ad889 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:48.970366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: WARNING py.warnings [None req-10860963-730b-4b2d-ba30-b0a58f7ad889 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:15:48.970366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: warnings.warn(msg) Jun 27 21:15:48.970366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: Jun 27 21:15:48.971252 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10860963-730b-4b2d-ba30-b0a58f7ad889 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:48.977978 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-fef7e31c-4f81-471b-a830-c2dbd47ef427 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:48.979217 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fef7e31c-4f81-471b-a830-c2dbd47ef427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:48.985117 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 770/1559] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:15:48.992407 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bcf6c-a436-4308-9fab-189161b82bee None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.992407 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bcf6c-a436-4308-9fab-189161b82bee None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.992407 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bcf6c-a436-4308-9fab-189161b82bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.995072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 790/1560] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:48.996283 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef7e31c-4f81-471b-a830-c2dbd47ef427 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.005450 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-4e8bcf6c-a436-4308-9fab-189161b82bee None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:49.006194 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef7e31c-4f81-471b-a830-c2dbd47ef427 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.006657 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4e8bcf6c-a436-4308-9fab-189161b82bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:49.010649 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad6de0-9c0a-4cc1-aac9-c957c46bad16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.011298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad6de0-9c0a-4cc1-aac9-c957c46bad16 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.011576 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad6de0-9c0a-4cc1-aac9-c957c46bad16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.011889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad6de0-9c0a-4cc1-aac9-c957c46bad16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.012570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad6de0-9c0a-4cc1-aac9-c957c46bad16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.013624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-b9ad6de0-9c0a-4cc1-aac9-c957c46bad16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:15:49.013624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 27 21:15:49.013624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: Jun 27 21:15:49.014431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 771/1561] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:49.014810 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad6de0-9c0a-4cc1-aac9-c957c46bad16 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.015644 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0521693-d64a-43c1-995b-bbb14ba09635 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.015935 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0521693-d64a-43c1-995b-bbb14ba09635 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.016214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0521693-d64a-43c1-995b-bbb14ba09635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.036194 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d0521693-d64a-43c1-995b-bbb14ba09635 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:49.037436 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d0521693-d64a-43c1-995b-bbb14ba09635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:49.042569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 791/1562] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:48 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:49.043400 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad6de0-9c0a-4cc1-aac9-c957c46bad16 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.045469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 772/1563] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:49.049079 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0521693-d64a-43c1-995b-bbb14ba09635 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.057997 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0521693-d64a-43c1-995b-bbb14ba09635 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.094552 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:49.106677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad6de0-9c0a-4cc1-aac9-c957c46bad16 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.106895 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:49.112196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a606010-5b13-486c-93f1-8a8d30ef93c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.113090 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a606010-5b13-486c-93f1-8a8d30ef93c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.113376 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a606010-5b13-486c-93f1-8a8d30ef93c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.113653 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a606010-5b13-486c-93f1-8a8d30ef93c7 admin admin] PATH_INFO: `/v3/projects/0d49eb39209b4612aeb8b13eff382b84/users/14307670704943768a05d7d6e5a7318b/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.114679 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadb92d-4a3b-4939-9250-9d7615857e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.115361 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadb92d-4a3b-4939-9250-9d7615857e7d admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.115865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadb92d-4a3b-4939-9250-9d7615857e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.116144 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadb92d-4a3b-4939-9250-9d7615857e7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.116876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadb92d-4a3b-4939-9250-9d7615857e7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:15:49.117192 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadb92d-4a3b-4939-9250-9d7615857e7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.118812 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadb92d-4a3b-4939-9250-9d7615857e7d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.128486 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a606010-5b13-486c-93f1-8a8d30ef93c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d49eb39209b4612aeb8b13eff382b84, user_id=14307670704943768a05d7d6e5a7318b, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.129567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-2a606010-5b13-486c-93f1-8a8d30ef93c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:15:49.129567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 27 21:15:49.129567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: Jun 27 21:15:49.131216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 773/1564] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:15:49.131520 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a606010-5b13-486c-93f1-8a8d30ef93c7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.132233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadb92d-4a3b-4939-9250-9d7615857e7d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.142920 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadb92d-4a3b-4939-9250-9d7615857e7d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.146482 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 792/1565] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:49 2022] PUT /identity/v3/projects/0d49eb39209b4612aeb8b13eff382b84/users/14307670704943768a05d7d6e5a7318b/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:49.147277 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a606010-5b13-486c-93f1-8a8d30ef93c7 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.199141 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bafd3-115a-4d4c-b5c8-8a02c8cd56cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.199850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bafd3-115a-4d4c-b5c8-8a02c8cd56cf admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.200133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bafd3-115a-4d4c-b5c8-8a02c8cd56cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.200416 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bafd3-115a-4d4c-b5c8-8a02c8cd56cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.201126 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518bafd3-115a-4d4c-b5c8-8a02c8cd56cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:15:49.201437 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518bafd3-115a-4d4c-b5c8-8a02c8cd56cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.203081 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518bafd3-115a-4d4c-b5c8-8a02c8cd56cf admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.211334 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a606010-5b13-486c-93f1-8a8d30ef93c7 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.215199 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 774/1566] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:15:49.222695 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bafd3-115a-4d4c-b5c8-8a02c8cd56cf admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.222816 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738860b1-6004-4cab-b390-b2efb442b683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.222816 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738860b1-6004-4cab-b390-b2efb442b683 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.222816 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738860b1-6004-4cab-b390-b2efb442b683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.222816 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738860b1-6004-4cab-b390-b2efb442b683 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.223192 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738860b1-6004-4cab-b390-b2efb442b683 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.223192 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738860b1-6004-4cab-b390-b2efb442b683 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.228520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bafd3-115a-4d4c-b5c8-8a02c8cd56cf admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.233509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e7a21-21fe-42dc-a251-69349ff5c45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.234157 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e7a21-21fe-42dc-a251-69349ff5c45e admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.234510 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e7a21-21fe-42dc-a251-69349ff5c45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.234705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e7a21-21fe-42dc-a251-69349ff5c45e admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.235533 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e7a21-21fe-42dc-a251-69349ff5c45e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:15:49.235961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e7a21-21fe-42dc-a251-69349ff5c45e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.237730 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e7a21-21fe-42dc-a251-69349ff5c45e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.249333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 793/1567] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:49 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:49.249333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738860b1-6004-4cab-b390-b2efb442b683 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.255487 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 775/1568] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:15:49.255487 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e7a21-21fe-42dc-a251-69349ff5c45e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.264711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e7a21-21fe-42dc-a251-69349ff5c45e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.315213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738860b1-6004-4cab-b390-b2efb442b683 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.319940 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926269eb-98a2-4853-97f6-929ea5505888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.320121 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824b75d-b695-4dc7-aa6a-c4239f3016bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.320616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926269eb-98a2-4853-97f6-929ea5505888 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.320785 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824b75d-b695-4dc7-aa6a-c4239f3016bc admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.320901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926269eb-98a2-4853-97f6-929ea5505888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.321067 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824b75d-b695-4dc7-aa6a-c4239f3016bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.321167 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926269eb-98a2-4853-97f6-929ea5505888 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.321336 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824b75d-b695-4dc7-aa6a-c4239f3016bc admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.321877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926269eb-98a2-4853-97f6-929ea5505888 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:15:49.322061 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2824b75d-b695-4dc7-aa6a-c4239f3016bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.322199 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926269eb-98a2-4853-97f6-929ea5505888 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.323818 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926269eb-98a2-4853-97f6-929ea5505888 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.323939 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2824b75d-b695-4dc7-aa6a-c4239f3016bc admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.335950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 794/1569] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:15:49.336808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926269eb-98a2-4853-97f6-929ea5505888 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.346654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926269eb-98a2-4853-97f6-929ea5505888 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.351591 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427b3c9-b2ba-4046-bd41-2bc5d3f114e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.352282 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427b3c9-b2ba-4046-bd41-2bc5d3f114e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.352569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427b3c9-b2ba-4046-bd41-2bc5d3f114e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.352843 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427b3c9-b2ba-4046-bd41-2bc5d3f114e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.353545 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e427b3c9-b2ba-4046-bd41-2bc5d3f114e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:15:49.353868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e427b3c9-b2ba-4046-bd41-2bc5d3f114e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.357410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 776/1570] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:49 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:49.357410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824b75d-b695-4dc7-aa6a-c4239f3016bc admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.358013 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e427b3c9-b2ba-4046-bd41-2bc5d3f114e1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.367176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824b75d-b695-4dc7-aa6a-c4239f3016bc admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.369471 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 795/1571] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:15:49.370744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1379cd-f02d-4a59-99a0-903efbcc4cc4 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.370999 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1379cd-f02d-4a59-99a0-903efbcc4cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.371291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1379cd-f02d-4a59-99a0-903efbcc4cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.372326 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84b03e-7bc2-438d-8689-cb6ed6095eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.373035 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84b03e-7bc2-438d-8689-cb6ed6095eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.373266 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84b03e-7bc2-438d-8689-cb6ed6095eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.373536 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84b03e-7bc2-438d-8689-cb6ed6095eb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.374347 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a84b03e-7bc2-438d-8689-cb6ed6095eb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.376545 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a84b03e-7bc2-438d-8689-cb6ed6095eb8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.389068 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ea1379cd-f02d-4a59-99a0-903efbcc4cc4 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:49.390277 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ea1379cd-f02d-4a59-99a0-903efbcc4cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:49.398849 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 796/1572] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:49.399630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1379cd-f02d-4a59-99a0-903efbcc4cc4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.406407 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 777/1573] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:49 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:49.407863 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84b03e-7bc2-438d-8689-cb6ed6095eb8 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.411295 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1379cd-f02d-4a59-99a0-903efbcc4cc4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.440833 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:49.451572 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:49.470482 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e576ffc-05f2-423c-be3c-23a3d787cb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.471144 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e576ffc-05f2-423c-be3c-23a3d787cb2b admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.471426 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e576ffc-05f2-423c-be3c-23a3d787cb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.471727 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e576ffc-05f2-423c-be3c-23a3d787cb2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.472458 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e576ffc-05f2-423c-be3c-23a3d787cb2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.473252 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84b03e-7bc2-438d-8689-cb6ed6095eb8 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.474375 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e576ffc-05f2-423c-be3c-23a3d787cb2b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.477955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd620983-c14d-41da-9ec6-d99154e6607b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.478627 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd620983-c14d-41da-9ec6-d99154e6607b admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.478906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd620983-c14d-41da-9ec6-d99154e6607b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.479179 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd620983-c14d-41da-9ec6-d99154e6607b admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.479892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd620983-c14d-41da-9ec6-d99154e6607b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.481897 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd620983-c14d-41da-9ec6-d99154e6607b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.500743 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 797/1574] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:49 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:49.501603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e576ffc-05f2-423c-be3c-23a3d787cb2b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.503327 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 778/1575] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:49 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:49.504356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd620983-c14d-41da-9ec6-d99154e6607b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.515255 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd620983-c14d-41da-9ec6-d99154e6607b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.565951 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e576ffc-05f2-423c-be3c-23a3d787cb2b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.570450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652bc07-892c-4230-9916-a1170c29eb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.571139 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb240f-adf9-45c8-8d61-e133979f9f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.571273 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652bc07-892c-4230-9916-a1170c29eb90 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.571417 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652bc07-892c-4230-9916-a1170c29eb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.571715 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652bc07-892c-4230-9916-a1170c29eb90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.571911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb240f-adf9-45c8-8d61-e133979f9f41 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.572194 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb240f-adf9-45c8-8d61-e133979f9f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.572413 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652bc07-892c-4230-9916-a1170c29eb90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.572507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb240f-adf9-45c8-8d61-e133979f9f41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.573139 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb240f-adf9-45c8-8d61-e133979f9f41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.574353 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652bc07-892c-4230-9916-a1170c29eb90 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.575049 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb240f-adf9-45c8-8d61-e133979f9f41 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.604030 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 798/1576] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:49 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:49.604030 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652bc07-892c-4230-9916-a1170c29eb90 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.609227 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 779/1577] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:49 2022] POST /identity/v3/projects => generated 389 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:49.609227 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb240f-adf9-45c8-8d61-e133979f9f41 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.668081 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb240f-adf9-45c8-8d61-e133979f9f41 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.670350 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652bc07-892c-4230-9916-a1170c29eb90 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.672758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0709be-8122-4de4-8a40-0dc19ac9bed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.673422 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0709be-8122-4de4-8a40-0dc19ac9bed5 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.673701 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0709be-8122-4de4-8a40-0dc19ac9bed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.674036 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0709be-8122-4de4-8a40-0dc19ac9bed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.674667 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0709be-8122-4de4-8a40-0dc19ac9bed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.675368 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc6beb-a60a-4019-b4a5-8d0c1240e7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.676080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc6beb-a60a-4019-b4a5-8d0c1240e7b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.676359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc6beb-a60a-4019-b4a5-8d0c1240e7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.676625 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0709be-8122-4de4-8a40-0dc19ac9bed5 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.676727 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc6beb-a60a-4019-b4a5-8d0c1240e7b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.677346 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc6beb-a60a-4019-b4a5-8d0c1240e7b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.678956 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc6beb-a60a-4019-b4a5-8d0c1240e7b2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.689581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 799/1578] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:49.690471 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc6beb-a60a-4019-b4a5-8d0c1240e7b2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.704268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 780/1579] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:49 2022] POST /identity/v3/projects => generated 401 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:49.705089 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0709be-8122-4de4-8a40-0dc19ac9bed5 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.734358 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:49.745550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:49.749485 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc6beb-a60a-4019-b4a5-8d0c1240e7b2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.767547 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0709be-8122-4de4-8a40-0dc19ac9bed5 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.774612 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08590e4e-0027-4eea-bed3-72576c0be04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.775344 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08590e4e-0027-4eea-bed3-72576c0be04d admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.775695 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08590e4e-0027-4eea-bed3-72576c0be04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.775978 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08590e4e-0027-4eea-bed3-72576c0be04d admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.776677 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08590e4e-0027-4eea-bed3-72576c0be04d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.778633 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08590e4e-0027-4eea-bed3-72576c0be04d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.800227 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 781/1580] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:49 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:49.801005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08590e4e-0027-4eea-bed3-72576c0be04d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.804070 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665b8ff-65d7-49df-8447-bdd5415ca1d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.804737 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665b8ff-65d7-49df-8447-bdd5415ca1d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.805016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665b8ff-65d7-49df-8447-bdd5415ca1d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.805289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665b8ff-65d7-49df-8447-bdd5415ca1d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.812837 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08590e4e-0027-4eea-bed3-72576c0be04d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.815161 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665b8ff-65d7-49df-8447-bdd5415ca1d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.816689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665b8ff-65d7-49df-8447-bdd5415ca1d9 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.817907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff4873-b343-4285-bfc7-64af90c52c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.818551 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff4873-b343-4285-bfc7-64af90c52c61 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.818837 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff4873-b343-4285-bfc7-64af90c52c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.819112 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff4873-b343-4285-bfc7-64af90c52c61 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.819839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff4873-b343-4285-bfc7-64af90c52c61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.821751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff4873-b343-4285-bfc7-64af90c52c61 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.825902 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 800/1581] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:49.826686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665b8ff-65d7-49df-8447-bdd5415ca1d9 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.836071 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665b8ff-65d7-49df-8447-bdd5415ca1d9 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.840838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a8361-9421-42ae-8d2f-3af91454101b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.841498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a8361-9421-42ae-8d2f-3af91454101b admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.841777 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a8361-9421-42ae-8d2f-3af91454101b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.842048 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a8361-9421-42ae-8d2f-3af91454101b admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.843236 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a8361-9421-42ae-8d2f-3af91454101b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.844082 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 782/1582] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:49 2022] POST /identity/v3/users => generated 292 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:49.844420 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a8361-9421-42ae-8d2f-3af91454101b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.848013 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff4873-b343-4285-bfc7-64af90c52c61 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.856363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 801/1583] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:49.858909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff4873-b343-4285-bfc7-64af90c52c61 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.860154 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a8361-9421-42ae-8d2f-3af91454101b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.864583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559b5d2-6e39-40f5-9eb3-0dbf4e9ce2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.865260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559b5d2-6e39-40f5-9eb3-0dbf4e9ce2cb admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.865591 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559b5d2-6e39-40f5-9eb3-0dbf4e9ce2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.865805 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559b5d2-6e39-40f5-9eb3-0dbf4e9ce2cb admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.866497 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559b5d2-6e39-40f5-9eb3-0dbf4e9ce2cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.868478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559b5d2-6e39-40f5-9eb3-0dbf4e9ce2cb admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.871010 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a8361-9421-42ae-8d2f-3af91454101b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.876179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e3c4c-0611-478c-8e32-519047814d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.876998 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e3c4c-0611-478c-8e32-519047814d2f admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.877176 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e3c4c-0611-478c-8e32-519047814d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.877701 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e3c4c-0611-478c-8e32-519047814d2f admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.878442 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e3c4c-0611-478c-8e32-519047814d2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.880616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e3c4c-0611-478c-8e32-519047814d2f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.892608 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 783/1584] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:49 2022] POST /identity/v3/users => generated 286 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:49.897073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559b5d2-6e39-40f5-9eb3-0dbf4e9ce2cb admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.904344 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 802/1585] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:49 2022] POST /identity/v3/users => generated 295 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:49.904868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e3c4c-0611-478c-8e32-519047814d2f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.908590 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559b5d2-6e39-40f5-9eb3-0dbf4e9ce2cb admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.914351 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04230a6-c6cb-4943-9bdd-3ed7fec92601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.915062 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04230a6-c6cb-4943-9bdd-3ed7fec92601 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.915374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04230a6-c6cb-4943-9bdd-3ed7fec92601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.915743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04230a6-c6cb-4943-9bdd-3ed7fec92601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.916241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e3c4c-0611-478c-8e32-519047814d2f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.916497 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04230a6-c6cb-4943-9bdd-3ed7fec92601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.918264 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04230a6-c6cb-4943-9bdd-3ed7fec92601 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.921168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa2ff7-9f15-4621-90d9-0560d79e117f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.921928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa2ff7-9f15-4621-90d9-0560d79e117f admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.922192 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa2ff7-9f15-4621-90d9-0560d79e117f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.922461 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa2ff7-9f15-4621-90d9-0560d79e117f admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.923328 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa2ff7-9f15-4621-90d9-0560d79e117f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.925553 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa2ff7-9f15-4621-90d9-0560d79e117f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.931123 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 784/1586] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:49.932458 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04230a6-c6cb-4943-9bdd-3ed7fec92601 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.945744 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04230a6-c6cb-4943-9bdd-3ed7fec92601 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.949948 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 803/1587] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:49 2022] POST /identity/v3/users => generated 301 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:49.950785 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea31b3-3be0-4e21-9677-3f3ce4d2b35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.951493 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea31b3-3be0-4e21-9677-3f3ce4d2b35d admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.951801 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea31b3-3be0-4e21-9677-3f3ce4d2b35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.952088 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea31b3-3be0-4e21-9677-3f3ce4d2b35d admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.953009 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea31b3-3be0-4e21-9677-3f3ce4d2b35d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.953716 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa2ff7-9f15-4621-90d9-0560d79e117f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.956036 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea31b3-3be0-4e21-9677-3f3ce4d2b35d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.964146 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa2ff7-9f15-4621-90d9-0560d79e117f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.970485 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954038ce-056e-4dc7-97d5-95d63ad68401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:49.971199 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954038ce-056e-4dc7-97d5-95d63ad68401 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:49.971498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954038ce-056e-4dc7-97d5-95d63ad68401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:49.971791 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954038ce-056e-4dc7-97d5-95d63ad68401 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:49.972537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954038ce-056e-4dc7-97d5-95d63ad68401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:49.974172 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954038ce-056e-4dc7-97d5-95d63ad68401 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:49.982520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 785/1588] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:49.983880 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea31b3-3be0-4e21-9677-3f3ce4d2b35d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.991336 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 804/1589] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:49.993451 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954038ce-056e-4dc7-97d5-95d63ad68401 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:49.994637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea31b3-3be0-4e21-9677-3f3ce4d2b35d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:49.999936 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1427b-e853-4129-b1e4-e893e14f1224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:50.000989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1427b-e853-4129-b1e4-e893e14f1224 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.001338 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1427b-e853-4129-b1e4-e893e14f1224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.002305 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1427b-e853-4129-b1e4-e893e14f1224 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.003452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1427b-e853-4129-b1e4-e893e14f1224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:50.005467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1427b-e853-4129-b1e4-e893e14f1224 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:50.006238 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954038ce-056e-4dc7-97d5-95d63ad68401 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:50.014430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108b88b-42a2-41a3-9e3a-6db934ac71bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:50.015237 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108b88b-42a2-41a3-9e3a-6db934ac71bd admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.015755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108b88b-42a2-41a3-9e3a-6db934ac71bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.016101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108b88b-42a2-41a3-9e3a-6db934ac71bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.017457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108b88b-42a2-41a3-9e3a-6db934ac71bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:50.020095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108b88b-42a2-41a3-9e3a-6db934ac71bd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:50.027328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 786/1590] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:50.028175 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1427b-e853-4129-b1e4-e893e14f1224 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:50.040396 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 805/1591] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:50.041255 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108b88b-42a2-41a3-9e3a-6db934ac71bd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:50.043957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1427b-e853-4129-b1e4-e893e14f1224 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:50.052099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5628878-6159-4a57-b7b4-29c92def72bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:50.053196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5628878-6159-4a57-b7b4-29c92def72bf admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.053588 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5628878-6159-4a57-b7b4-29c92def72bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.054029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5628878-6159-4a57-b7b4-29c92def72bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.055353 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108b88b-42a2-41a3-9e3a-6db934ac71bd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:50.055843 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5628878-6159-4a57-b7b4-29c92def72bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:50.059832 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5628878-6159-4a57-b7b4-29c92def72bf admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:50.060618 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ad258-a7e0-4ac7-bca3-531bfe97d892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:50.061525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ad258-a7e0-4ac7-bca3-531bfe97d892 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.061837 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ad258-a7e0-4ac7-bca3-531bfe97d892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.062089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ad258-a7e0-4ac7-bca3-531bfe97d892 admin admin] PATH_INFO: `/v3/projects/1e275005751843ed8390794673be7bf9/users/1831abf2fa2449b0b339511da158ce18/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.075754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 787/1592] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:50.075754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5628878-6159-4a57-b7b4-29c92def72bf admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:50.079172 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ad258-a7e0-4ac7-bca3-531bfe97d892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e275005751843ed8390794673be7bf9, user_id=1831abf2fa2449b0b339511da158ce18, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:50.082441 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ad258-a7e0-4ac7-bca3-531bfe97d892 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:50.085064 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5628878-6159-4a57-b7b4-29c92def72bf admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:50.091102 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26620de5-263a-4751-b0bc-c780242eec58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:50.092571 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26620de5-263a-4751-b0bc-c780242eec58 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.093021 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26620de5-263a-4751-b0bc-c780242eec58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.093240 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26620de5-263a-4751-b0bc-c780242eec58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.093976 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26620de5-263a-4751-b0bc-c780242eec58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:50.095716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26620de5-263a-4751-b0bc-c780242eec58 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:50.098778 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 806/1593] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:50 2022] PUT /identity/v3/projects/1e275005751843ed8390794673be7bf9/users/1831abf2fa2449b0b339511da158ce18/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:50.099642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ad258-a7e0-4ac7-bca3-531bfe97d892 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:50.114094 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 788/1594] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:50.116567 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26620de5-263a-4751-b0bc-c780242eec58 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:50.130765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26620de5-263a-4751-b0bc-c780242eec58 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:50.166238 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ad258-a7e0-4ac7-bca3-531bfe97d892 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:50.170945 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc0ca8-3b57-46ad-b06a-73d66448fa6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:50.171645 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc0ca8-3b57-46ad-b06a-73d66448fa6f admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.171929 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc0ca8-3b57-46ad-b06a-73d66448fa6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.172201 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc0ca8-3b57-46ad-b06a-73d66448fa6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.173049 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc0ca8-3b57-46ad-b06a-73d66448fa6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:50.174714 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc0ca8-3b57-46ad-b06a-73d66448fa6f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:50.185281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 807/1595] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:50.189258 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc0ca8-3b57-46ad-b06a-73d66448fa6f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:50.195945 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:50.205125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc0ca8-3b57-46ad-b06a-73d66448fa6f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:50.212480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b47727-b422-44ae-99fa-d823b20e13e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:50.213896 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b47727-b422-44ae-99fa-d823b20e13e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.214593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b47727-b422-44ae-99fa-d823b20e13e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.215342 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b47727-b422-44ae-99fa-d823b20e13e6 admin admin] PATH_INFO: `/v3/projects/36fbf32e85d84d888f91e99337b3518b/users/4ed0920154434d74b84fae18deff3449/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.220707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab045866-8e63-4488-bb6d-cf97317e9309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:50.221754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab045866-8e63-4488-bb6d-cf97317e9309 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.222189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab045866-8e63-4488-bb6d-cf97317e9309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.222627 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab045866-8e63-4488-bb6d-cf97317e9309 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.223515 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab045866-8e63-4488-bb6d-cf97317e9309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:50.225328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab045866-8e63-4488-bb6d-cf97317e9309 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:50.234541 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b47727-b422-44ae-99fa-d823b20e13e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36fbf32e85d84d888f91e99337b3518b, user_id=4ed0920154434d74b84fae18deff3449, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:50.237126 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 789/1596] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:50.238148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab045866-8e63-4488-bb6d-cf97317e9309 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:50.241140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b47727-b422-44ae-99fa-d823b20e13e6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:50.250800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab045866-8e63-4488-bb6d-cf97317e9309 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:50.269698 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 808/1597] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:50 2022] PUT /identity/v3/projects/36fbf32e85d84d888f91e99337b3518b/users/4ed0920154434d74b84fae18deff3449/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:50.269698 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b47727-b422-44ae-99fa-d823b20e13e6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:50.299724 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:50.333581 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16249795-86cd-4f9b-8578-c2f1e459cdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:50.334320 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16249795-86cd-4f9b-8578-c2f1e459cdb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.334674 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16249795-86cd-4f9b-8578-c2f1e459cdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.335016 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16249795-86cd-4f9b-8578-c2f1e459cdb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.336482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16249795-86cd-4f9b-8578-c2f1e459cdb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:50.338223 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16249795-86cd-4f9b-8578-c2f1e459cdb3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:50.339399 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b47727-b422-44ae-99fa-d823b20e13e6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:50.344708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c70f3c-d55a-4d0b-8701-d0f4919d327c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:50.346018 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c70f3c-d55a-4d0b-8701-d0f4919d327c admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.346375 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c70f3c-d55a-4d0b-8701-d0f4919d327c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.346719 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c70f3c-d55a-4d0b-8701-d0f4919d327c admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.348374 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c70f3c-d55a-4d0b-8701-d0f4919d327c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:50.350102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c70f3c-d55a-4d0b-8701-d0f4919d327c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:50.364860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 790/1598] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:50.364860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9832e-5626-45c3-93a3-2a4f456d5515 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.364860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9832e-5626-45c3-93a3-2a4f456d5515 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.364860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9832e-5626-45c3-93a3-2a4f456d5515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.365763 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 809/1599] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:50.365763 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c70f3c-d55a-4d0b-8701-d0f4919d327c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:50.377326 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c70f3c-d55a-4d0b-8701-d0f4919d327c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:50.385585 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34dd7c-4149-4b6d-b0d3-039d9da70db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:50.386264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34dd7c-4149-4b6d-b0d3-039d9da70db6 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.386640 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34dd7c-4149-4b6d-b0d3-039d9da70db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.386952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34dd7c-4149-4b6d-b0d3-039d9da70db6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.387746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d34dd7c-4149-4b6d-b0d3-039d9da70db6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:50.389418 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d34dd7c-4149-4b6d-b0d3-039d9da70db6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:50.397746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-29f9832e-5626-45c3-93a3-2a4f456d5515 None None] MFA Rules not processed for user `1831abf2fa2449b0b339511da158ce18`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:50.399428 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-29f9832e-5626-45c3-93a3-2a4f456d5515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:50.402381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 810/1600] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:50.407041 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34dd7c-4149-4b6d-b0d3-039d9da70db6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:50.418243 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34dd7c-4149-4b6d-b0d3-039d9da70db6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:50.424028 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e554351-64f9-4f9c-9b3e-25e416ee18ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:50.424935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e554351-64f9-4f9c-9b3e-25e416ee18ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.425216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e554351-64f9-4f9c-9b3e-25e416ee18ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.425488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e554351-64f9-4f9c-9b3e-25e416ee18ae admin admin] PATH_INFO: `/v3/projects/a60658a4e6084eba8bff08bef89a8b8d/users/f56292d1ba174a02b15853d61f340d57/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.440445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e554351-64f9-4f9c-9b3e-25e416ee18ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=a60658a4e6084eba8bff08bef89a8b8d, user_id=f56292d1ba174a02b15853d61f340d57, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:50.443123 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e554351-64f9-4f9c-9b3e-25e416ee18ae admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:50.467755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 811/1601] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:50 2022] PUT /identity/v3/projects/a60658a4e6084eba8bff08bef89a8b8d/users/f56292d1ba174a02b15853d61f340d57/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:50.467755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e554351-64f9-4f9c-9b3e-25e416ee18ae admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:50.479824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 791/1602] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:50 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:50.481979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5a0e2-68ea-4b67-bec0-5f42bddf4018 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.482335 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5a0e2-68ea-4b67-bec0-5f42bddf4018 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.482682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5a0e2-68ea-4b67-bec0-5f42bddf4018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.513945 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-f5c5a0e2-68ea-4b67-bec0-5f42bddf4018 None None] MFA Rules not processed for user `4ed0920154434d74b84fae18deff3449`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:50.515348 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f5c5a0e2-68ea-4b67-bec0-5f42bddf4018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:50.537862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e554351-64f9-4f9c-9b3e-25e416ee18ae admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:50.543199 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67266df-8863-4430-82dc-37ae03b2d878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:50.544270 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67266df-8863-4430-82dc-37ae03b2d878 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.544619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67266df-8863-4430-82dc-37ae03b2d878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.544995 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67266df-8863-4430-82dc-37ae03b2d878 admin admin] PATH_INFO: `/v3/projects/0b5921bd9539401d9a235ccde185632b/users/30cbde26a1964f22902c24917006fb90/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.563645 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67266df-8863-4430-82dc-37ae03b2d878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b5921bd9539401d9a235ccde185632b, user_id=30cbde26a1964f22902c24917006fb90, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:50.567143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67266df-8863-4430-82dc-37ae03b2d878 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:50.604027 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 812/1603] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:50 2022] PUT /identity/v3/projects/0b5921bd9539401d9a235ccde185632b/users/30cbde26a1964f22902c24917006fb90/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:50.604027 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67266df-8863-4430-82dc-37ae03b2d878 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:50.604353 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 792/1604] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:50.604502 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5a0e2-68ea-4b67-bec0-5f42bddf4018 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:50.629060 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5a0e2-68ea-4b67-bec0-5f42bddf4018 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:50.636919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:50.691280 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:50.706058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67266df-8863-4430-82dc-37ae03b2d878 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:50.712177 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb6406-7e43-4818-962f-9008036f28e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:50.712696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c20dea-0cc5-4f58-9bf1-b92544c52e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:50.713127 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb6406-7e43-4818-962f-9008036f28e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.713480 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb6406-7e43-4818-962f-9008036f28e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.713689 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb6406-7e43-4818-962f-9008036f28e2 admin admin] PATH_INFO: `/v3/projects/6b75fb163caa47cd9b72e047e9e050e2/users/74b2295521bc45509ef6b68c1885ac6e/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.713839 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c20dea-0cc5-4f58-9bf1-b92544c52e70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.714201 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c20dea-0cc5-4f58-9bf1-b92544c52e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.714547 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c20dea-0cc5-4f58-9bf1-b92544c52e70 admin admin] PATH_INFO: `/v3/projects/474443a05dd94540931e136c5e5abc54/users/0dcfc6a4bdfa474a8aebfa593b466684/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.733636 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c20dea-0cc5-4f58-9bf1-b92544c52e70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=474443a05dd94540931e136c5e5abc54, user_id=0dcfc6a4bdfa474a8aebfa593b466684, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:50.737150 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb6406-7e43-4818-962f-9008036f28e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b75fb163caa47cd9b72e047e9e050e2, user_id=74b2295521bc45509ef6b68c1885ac6e, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:50.737831 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c20dea-0cc5-4f58-9bf1-b92544c52e70 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:50.739819 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb6406-7e43-4818-962f-9008036f28e2 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:50.756474 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 813/1605] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:50 2022] PUT /identity/v3/projects/474443a05dd94540931e136c5e5abc54/users/0dcfc6a4bdfa474a8aebfa593b466684/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:50.757621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c20dea-0cc5-4f58-9bf1-b92544c52e70 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:50.763090 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 793/1606] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:50 2022] PUT /identity/v3/projects/6b75fb163caa47cd9b72e047e9e050e2/users/74b2295521bc45509ef6b68c1885ac6e/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:50.763090 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8814a-68d4-4783-b5d3-847b581cb25a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.763090 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8814a-68d4-4783-b5d3-847b581cb25a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.763090 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8814a-68d4-4783-b5d3-847b581cb25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.796145 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-bff8814a-68d4-4783-b5d3-847b581cb25a None None] MFA Rules not processed for user `f56292d1ba174a02b15853d61f340d57`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:50.797616 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-bff8814a-68d4-4783-b5d3-847b581cb25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:50.858099 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c20dea-0cc5-4f58-9bf1-b92544c52e70 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:50.887380 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 794/1607] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:50.888532 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8814a-68d4-4783-b5d3-847b581cb25a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:50.928125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb73d65-fc4b-4125-944c-6da72388e439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:50.929271 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb73d65-fc4b-4125-944c-6da72388e439 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.929633 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb73d65-fc4b-4125-944c-6da72388e439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.929978 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb73d65-fc4b-4125-944c-6da72388e439 admin admin] PATH_INFO: `/v3/projects/2de03c4861a6487d9465abdaa4f3bc53/users/c6aafd1e5fef4e76a6e73879c775fd8d/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.946852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb73d65-fc4b-4125-944c-6da72388e439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2de03c4861a6487d9465abdaa4f3bc53, user_id=c6aafd1e5fef4e76a6e73879c775fd8d, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:50.949661 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb73d65-fc4b-4125-944c-6da72388e439 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:50.962366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8814a-68d4-4783-b5d3-847b581cb25a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:50.969979 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 814/1608] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:50 2022] PUT /identity/v3/projects/2de03c4861a6487d9465abdaa4f3bc53/users/c6aafd1e5fef4e76a6e73879c775fd8d/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:50.972431 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828591e-7d18-4b8d-8b33-225edc42ea1a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.973058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828591e-7d18-4b8d-8b33-225edc42ea1a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.973479 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828591e-7d18-4b8d-8b33-225edc42ea1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:51.004726 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a828591e-7d18-4b8d-8b33-225edc42ea1a None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:51.006646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a828591e-7d18-4b8d-8b33-225edc42ea1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:51.041406 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410fb66-cf48-41f7-b98c-4095985f1de8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:51.042715 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410fb66-cf48-41f7-b98c-4095985f1de8 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:51.045403 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410fb66-cf48-41f7-b98c-4095985f1de8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:51.045403 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410fb66-cf48-41f7-b98c-4095985f1de8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:51.060519 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:51.084783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:51.107809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 815/1609] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:51.107809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901672b8-678f-4bf9-b76c-9790d85d573a None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:51.107809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901672b8-678f-4bf9-b76c-9790d85d573a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:51.107809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901672b8-678f-4bf9-b76c-9790d85d573a None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:51.111691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 816/1610] 10.222.0.27 () {54 vars in 961 bytes} [Mon Jun 27 21:15:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:15:51.114915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934a689-2026-4b57-bfa2-016a51840aa5 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:51.116882 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934a689-2026-4b57-bfa2-016a51840aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:51.117171 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934a689-2026-4b57-bfa2-016a51840aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:51.127251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410fb66-cf48-41f7-b98c-4095985f1de8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:51.129798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410fb66-cf48-41f7-b98c-4095985f1de8 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:51.145733 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 795/1611] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:50 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:51.147529 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab9f06-5932-477a-b6e2-b10662778dbf None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:51.147910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab9f06-5932-477a-b6e2-b10662778dbf None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:51.148301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab9f06-5932-477a-b6e2-b10662778dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:51.152039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6934a689-2026-4b57-bfa2-016a51840aa5 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:51.153842 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6934a689-2026-4b57-bfa2-016a51840aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:51.164876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 817/1612] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:51.166038 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934a689-2026-4b57-bfa2-016a51840aa5 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:51.179913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934a689-2026-4b57-bfa2-016a51840aa5 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:51.188141 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca7697-3ade-4447-8626-e8e8352e4037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:51.189015 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca7697-3ade-4447-8626-e8e8352e4037 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:51.189291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca7697-3ade-4447-8626-e8e8352e4037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:51.189569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca7697-3ade-4447-8626-e8e8352e4037 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:51.190538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca7697-3ade-4447-8626-e8e8352e4037 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:51.192831 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca7697-3ade-4447-8626-e8e8352e4037 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:51.198724 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-8cab9f06-5932-477a-b6e2-b10662778dbf None None] MFA Rules not processed for user `0dcfc6a4bdfa474a8aebfa593b466684`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:51.201038 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8cab9f06-5932-477a-b6e2-b10662778dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:51.237224 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 818/1613] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:51 2022] POST /identity/v3/projects => generated 381 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:51.238609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de3704-8ee3-424b-a0dd-ef44dc40209f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:51.238904 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de3704-8ee3-424b-a0dd-ef44dc40209f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:51.243896 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de3704-8ee3-424b-a0dd-ef44dc40209f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:51.300417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 796/1614] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:51.302175 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be298848-14ee-4d94-a0cb-a4a1bbfe5411 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:51.302621 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be298848-14ee-4d94-a0cb-a4a1bbfe5411 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:51.303059 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be298848-14ee-4d94-a0cb-a4a1bbfe5411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:51.310409 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d9de3704-8ee3-424b-a0dd-ef44dc40209f None None] MFA Rules not processed for user `c6aafd1e5fef4e76a6e73879c775fd8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:51.314482 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d9de3704-8ee3-424b-a0dd-ef44dc40209f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:51.331034 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-be298848-14ee-4d94-a0cb-a4a1bbfe5411 None None] MFA Rules not processed for user `8d7fe01ef64f46f6a0e77d5945219b33`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:51.332461 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-be298848-14ee-4d94-a0cb-a4a1bbfe5411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:51.440557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 819/1615] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:51.444836 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de3704-8ee3-424b-a0dd-ef44dc40209f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:51.465621 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 797/1616] 10.222.0.27 () {58 vars in 1058 bytes} [Mon Jun 27 21:15:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:51.466267 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be298848-14ee-4d94-a0cb-a4a1bbfe5411 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:51.469118 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-be298848-14ee-4d94-a0cb-a4a1bbfe5411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:51.582379 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be298848-14ee-4d94-a0cb-a4a1bbfe5411 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:51.591377 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75d275-913c-4337-bfef-4c53e7e4a87b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:51.592421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75d275-913c-4337-bfef-4c53e7e4a87b service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:51.593069 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75d275-913c-4337-bfef-4c53e7e4a87b service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:51.593511 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75d275-913c-4337-bfef-4c53e7e4a87b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:51.595147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de3704-8ee3-424b-a0dd-ef44dc40209f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:51.599841 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82739bb-ceb1-4217-9e95-689a5f4d6b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:51.600612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82739bb-ceb1-4217-9e95-689a5f4d6b85 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:51.601123 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82739bb-ceb1-4217-9e95-689a5f4d6b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:51.601411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82739bb-ceb1-4217-9e95-689a5f4d6b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:51.603819 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:51.690690 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:51.702369 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75d275-913c-4337-bfef-4c53e7e4a87b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:51.703694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75d275-913c-4337-bfef-4c53e7e4a87b service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:51.717136 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82739bb-ceb1-4217-9e95-689a5f4d6b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:51.719302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82739bb-ceb1-4217-9e95-689a5f4d6b85 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:51.722415 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 798/1617] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:51.737426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75d275-913c-4337-bfef-4c53e7e4a87b service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:51.740725 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 820/1618] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:51.742131 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352aef6-6af0-472b-bf94-f3e68418b278 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:51.742417 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352aef6-6af0-472b-bf94-f3e68418b278 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:51.742700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352aef6-6af0-472b-bf94-f3e68418b278 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:51.744856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 821/1619] 10.222.0.27 () {54 vars in 961 bytes} [Mon Jun 27 21:15:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:15:51.755139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352aef6-6af0-472b-bf94-f3e68418b278 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:51.777667 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75d275-913c-4337-bfef-4c53e7e4a87b service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:51.783626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f644e-8986-4df5-a587-07a462ec123d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:51.784552 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f644e-8986-4df5-a587-07a462ec123d admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:51.784898 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f644e-8986-4df5-a587-07a462ec123d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:51.785182 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f644e-8986-4df5-a587-07a462ec123d admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:51.786227 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f644e-8986-4df5-a587-07a462ec123d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:51.788117 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f644e-8986-4df5-a587-07a462ec123d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:51.809187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352aef6-6af0-472b-bf94-f3e68418b278 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:51.822892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af94e17-c5a2-42bb-8539-8d7f28a21afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:51.823079 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af94e17-c5a2-42bb-8539-8d7f28a21afe admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:51.823297 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af94e17-c5a2-42bb-8539-8d7f28a21afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:51.823546 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af94e17-c5a2-42bb-8539-8d7f28a21afe admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:51.824288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af94e17-c5a2-42bb-8539-8d7f28a21afe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:51.826386 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af94e17-c5a2-42bb-8539-8d7f28a21afe admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:51.913842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 799/1620] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:51 2022] POST /identity/v3/users => generated 297 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:51.913842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f644e-8986-4df5-a587-07a462ec123d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:51.913842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f644e-8986-4df5-a587-07a462ec123d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:51.915336 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 822/1621] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:51 2022] POST /identity/v3/projects => generated 403 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:51.915336 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4abe41-6cc2-457c-b4af-fbf5c97709b8 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:51.915336 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4abe41-6cc2-457c-b4af-fbf5c97709b8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:51.915336 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4abe41-6cc2-457c-b4af-fbf5c97709b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:51.915675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c1495-ac55-4b33-8ceb-80d645809dc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:51.915675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c1495-ac55-4b33-8ceb-80d645809dc0 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:51.915675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c1495-ac55-4b33-8ceb-80d645809dc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:51.915675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c1495-ac55-4b33-8ceb-80d645809dc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:51.936164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ce4abe41-6cc2-457c-b4af-fbf5c97709b8 None None] MFA Rules not processed for user `8d7fe01ef64f46f6a0e77d5945219b33`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:51.937416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ce4abe41-6cc2-457c-b4af-fbf5c97709b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:52.019735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c1495-ac55-4b33-8ceb-80d645809dc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:52.021676 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c1495-ac55-4b33-8ceb-80d645809dc0 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:52.033575 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 800/1622] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:52.045771 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c1495-ac55-4b33-8ceb-80d645809dc0 service nova] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:52.072442 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c1495-ac55-4b33-8ceb-80d645809dc0 service nova] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:52.128752 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 823/1623] 10.222.0.27 () {58 vars in 1058 bytes} [Mon Jun 27 21:15:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:52.129824 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4abe41-6cc2-457c-b4af-fbf5c97709b8 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:52.203052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a59dc2-0866-48f4-a0f1-1f445b1fefee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:52.203962 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a59dc2-0866-48f4-a0f1-1f445b1fefee admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:52.204541 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a59dc2-0866-48f4-a0f1-1f445b1fefee admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:52.204919 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a59dc2-0866-48f4-a0f1-1f445b1fefee admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:52.205778 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a59dc2-0866-48f4-a0f1-1f445b1fefee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:52.208059 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a59dc2-0866-48f4-a0f1-1f445b1fefee admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:52.234473 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 801/1624] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:52.235764 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a59dc2-0866-48f4-a0f1-1f445b1fefee admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:52.259981 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a59dc2-0866-48f4-a0f1-1f445b1fefee admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:52.263447 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4abe41-6cc2-457c-b4af-fbf5c97709b8 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:52.278106 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4846a9-0b02-411a-9f20-08089daa520f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:52.278824 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4846a9-0b02-411a-9f20-08089daa520f admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:52.279435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4846a9-0b02-411a-9f20-08089daa520f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:52.279435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4846a9-0b02-411a-9f20-08089daa520f admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:52.280076 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4846a9-0b02-411a-9f20-08089daa520f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:52.282351 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4846a9-0b02-411a-9f20-08089daa520f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:52.289209 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1b6ce-e0c9-44af-8433-3fc685806d31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:52.289877 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1b6ce-e0c9-44af-8433-3fc685806d31 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:52.290236 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1b6ce-e0c9-44af-8433-3fc685806d31 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:52.290445 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1b6ce-e0c9-44af-8433-3fc685806d31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:52.318051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:52.327656 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 824/1625] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:52 2022] POST /identity/v3/users => generated 308 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:52.328649 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4846a9-0b02-411a-9f20-08089daa520f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:52.351105 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4846a9-0b02-411a-9f20-08089daa520f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:52.361879 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b9925-f7a2-4081-a8eb-a97e59d8c08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:52.362914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b9925-f7a2-4081-a8eb-a97e59d8c08e admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:52.364643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b9925-f7a2-4081-a8eb-a97e59d8c08e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:52.364643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b9925-f7a2-4081-a8eb-a97e59d8c08e admin admin] PATH_INFO: `/v3/projects/c1ee56e38d3e44e791b9b82137cba69e/users/a28fd5319e674c5d8367a0893425702a/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:52.400487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b9925-f7a2-4081-a8eb-a97e59d8c08e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1ee56e38d3e44e791b9b82137cba69e, user_id=a28fd5319e674c5d8367a0893425702a, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:52.405588 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b9925-f7a2-4081-a8eb-a97e59d8c08e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:52.414371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1b6ce-e0c9-44af-8433-3fc685806d31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:52.416956 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1b6ce-e0c9-44af-8433-3fc685806d31 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:52.436493 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 825/1626] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:52 2022] PUT /identity/v3/projects/c1ee56e38d3e44e791b9b82137cba69e/users/a28fd5319e674c5d8367a0893425702a/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:52.455787 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b9925-f7a2-4081-a8eb-a97e59d8c08e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:52.456446 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 802/1627] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:52.456446 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1b6ce-e0c9-44af-8433-3fc685806d31 service nova] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:52.477699 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1b6ce-e0c9-44af-8433-3fc685806d31 service nova] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:52.491521 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:52.578039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b9925-f7a2-4081-a8eb-a97e59d8c08e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:52.587519 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21228094-59b3-4baf-b8d4-e1a4ac80dcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:52.588292 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21228094-59b3-4baf-b8d4-e1a4ac80dcd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:52.588583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21228094-59b3-4baf-b8d4-e1a4ac80dcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:52.588862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21228094-59b3-4baf-b8d4-e1a4ac80dcd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:52.590213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21228094-59b3-4baf-b8d4-e1a4ac80dcd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:52.592591 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21228094-59b3-4baf-b8d4-e1a4ac80dcd6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:52.627171 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 826/1628] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:52.630723 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6717010-7688-4005-a4bc-4520339997a2 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:52.631101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6717010-7688-4005-a4bc-4520339997a2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:52.631691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6717010-7688-4005-a4bc-4520339997a2 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:52.634839 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 827/1629] 10.222.0.27 () {54 vars in 912 bytes} [Mon Jun 27 21:15:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:15:52.637282 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6717010-7688-4005-a4bc-4520339997a2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:52.651686 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c414e00-9efc-48f3-8864-ec6c537d0589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:52.652806 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c414e00-9efc-48f3-8864-ec6c537d0589 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:52.654339 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6717010-7688-4005-a4bc-4520339997a2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:52.654639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c414e00-9efc-48f3-8864-ec6c537d0589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:52.655595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c414e00-9efc-48f3-8864-ec6c537d0589 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:52.657058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c414e00-9efc-48f3-8864-ec6c537d0589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:52.661097 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c414e00-9efc-48f3-8864-ec6c537d0589 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:52.661241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44453f00-242b-498f-b59c-c29b442394f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:52.662173 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44453f00-242b-498f-b59c-c29b442394f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:52.662468 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44453f00-242b-498f-b59c-c29b442394f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:52.662745 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44453f00-242b-498f-b59c-c29b442394f2 admin admin] PATH_INFO: `/v3/projects/70cc2f2f62b04ffabb6d5d341ea80244/users/c8e2121683a44bc288197ef2b668ef0b/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:52.680991 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44453f00-242b-498f-b59c-c29b442394f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70cc2f2f62b04ffabb6d5d341ea80244, user_id=c8e2121683a44bc288197ef2b668ef0b, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:52.684122 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44453f00-242b-498f-b59c-c29b442394f2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:52.695949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 803/1630] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:52.706197 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed5b29-9008-454f-8df4-7b2a1ab2cadb None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:52.708832 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed5b29-9008-454f-8df4-7b2a1ab2cadb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:52.709244 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed5b29-9008-454f-8df4-7b2a1ab2cadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:52.713183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 828/1631] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:52 2022] PUT /identity/v3/projects/70cc2f2f62b04ffabb6d5d341ea80244/users/c8e2121683a44bc288197ef2b668ef0b/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:52.723304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44453f00-242b-498f-b59c-c29b442394f2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:52.754954 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-62ed5b29-9008-454f-8df4-7b2a1ab2cadb None None] MFA Rules not processed for user `8d7fe01ef64f46f6a0e77d5945219b33`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:52.756417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-62ed5b29-9008-454f-8df4-7b2a1ab2cadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:52.856895 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44453f00-242b-498f-b59c-c29b442394f2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:52.870091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7fff2-83d9-42cb-8615-38fa5b11c9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:52.871015 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7fff2-83d9-42cb-8615-38fa5b11c9e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:52.871297 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7fff2-83d9-42cb-8615-38fa5b11c9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:52.871574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7fff2-83d9-42cb-8615-38fa5b11c9e3 admin admin] PATH_INFO: `/v3/projects/c1ee56e38d3e44e791b9b82137cba69e/users/a28fd5319e674c5d8367a0893425702a/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:52.876926 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7fff2-83d9-42cb-8615-38fa5b11c9e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1ee56e38d3e44e791b9b82137cba69e, user_id=a28fd5319e674c5d8367a0893425702a, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:52.879353 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 804/1632] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Jun 27 21:15:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:52.879712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7fff2-83d9-42cb-8615-38fa5b11c9e3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:52.885864 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed5b29-9008-454f-8df4-7b2a1ab2cadb None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:52.898358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed5b29-9008-454f-8df4-7b2a1ab2cadb None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:52.910661 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 829/1633] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:52 2022] PUT /identity/v3/projects/c1ee56e38d3e44e791b9b82137cba69e/users/a28fd5319e674c5d8367a0893425702a/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:52.911640 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7fff2-83d9-42cb-8615-38fa5b11c9e3 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:52.999357 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:53.019141 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba425729-dd19-4394-a87c-dea240d4555a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:53.020299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba425729-dd19-4394-a87c-dea240d4555a admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:53.020770 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba425729-dd19-4394-a87c-dea240d4555a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:53.021177 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba425729-dd19-4394-a87c-dea240d4555a admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:53.022061 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba425729-dd19-4394-a87c-dea240d4555a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:53.024899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba425729-dd19-4394-a87c-dea240d4555a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:53.050446 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 805/1634] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:53.058042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fc1a1-a2e1-4718-9907-3c35f403faac None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:53.058518 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fc1a1-a2e1-4718-9907-3c35f403faac None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:53.058956 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fc1a1-a2e1-4718-9907-3c35f403faac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:53.094810 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7fff2-83d9-42cb-8615-38fa5b11c9e3 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:53.109869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b27b0-4f58-4960-98df-049eb8fccf30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:53.110813 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b27b0-4f58-4960-98df-049eb8fccf30 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:53.111245 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b27b0-4f58-4960-98df-049eb8fccf30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:53.111710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b27b0-4f58-4960-98df-049eb8fccf30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:53.120102 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-2d9fc1a1-a2e1-4718-9907-3c35f403faac None None] MFA Rules not processed for user `a28fd5319e674c5d8367a0893425702a`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:53.120102 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2d9fc1a1-a2e1-4718-9907-3c35f403faac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:53.163689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:53.208365 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b27b0-4f58-4960-98df-049eb8fccf30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:53.211308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b27b0-4f58-4960-98df-049eb8fccf30 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:53.225354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 830/1635] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:53.226873 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f19512-a4da-4d5c-a138-b9aa39ff3c49 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:53.227176 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f19512-a4da-4d5c-a138-b9aa39ff3c49 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:53.227444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f19512-a4da-4d5c-a138-b9aa39ff3c49 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:53.233459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 831/1636] 10.222.0.27 () {54 vars in 912 bytes} [Mon Jun 27 21:15:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:15:53.234336 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f19512-a4da-4d5c-a138-b9aa39ff3c49 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:53.248179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f19512-a4da-4d5c-a138-b9aa39ff3c49 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:53.252974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34157472-2238-4cf3-b623-18512c6c341e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:53.253999 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34157472-2238-4cf3-b623-18512c6c341e admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:53.254416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34157472-2238-4cf3-b623-18512c6c341e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:53.254707 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34157472-2238-4cf3-b623-18512c6c341e admin admin] PATH_INFO: `/v3/projects/70cc2f2f62b04ffabb6d5d341ea80244/users/c8e2121683a44bc288197ef2b668ef0b/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:53.257467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34157472-2238-4cf3-b623-18512c6c341e admin admin] RBAC: Authorizing `identity:create_grant(project_id=70cc2f2f62b04ffabb6d5d341ea80244, user_id=c8e2121683a44bc288197ef2b668ef0b, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:53.260801 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34157472-2238-4cf3-b623-18512c6c341e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:53.276053 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 806/1637] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:53 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:53.277602 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffac505-570f-4e0e-b7de-8b29762b6e5a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:53.277969 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffac505-570f-4e0e-b7de-8b29762b6e5a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:53.278327 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffac505-570f-4e0e-b7de-8b29762b6e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:53.287244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 832/1638] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:53 2022] PUT /identity/v3/projects/70cc2f2f62b04ffabb6d5d341ea80244/users/c8e2121683a44bc288197ef2b668ef0b/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:53.298179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34157472-2238-4cf3-b623-18512c6c341e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:53.336250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-cffac505-570f-4e0e-b7de-8b29762b6e5a None None] MFA Rules not processed for user `8d7fe01ef64f46f6a0e77d5945219b33`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:53.337811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-cffac505-570f-4e0e-b7de-8b29762b6e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:53.403144 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34157472-2238-4cf3-b623-18512c6c341e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:53.430365 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863be73-11bf-42de-b65b-6f66178815d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:53.431058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863be73-11bf-42de-b65b-6f66178815d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:53.432555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863be73-11bf-42de-b65b-6f66178815d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:53.433391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863be73-11bf-42de-b65b-6f66178815d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:53.449486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 807/1639] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Jun 27 21:15:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:53.452018 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef38293-0b74-41f3-8a2a-cf955531c07c None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:53.452794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef38293-0b74-41f3-8a2a-cf955531c07c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:53.453172 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef38293-0b74-41f3-8a2a-cf955531c07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:53.529841 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-cef38293-0b74-41f3-8a2a-cf955531c07c None None] MFA Rules not processed for user `c8e2121683a44bc288197ef2b668ef0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:53.530677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863be73-11bf-42de-b65b-6f66178815d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:53.532435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863be73-11bf-42de-b65b-6f66178815d2 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:53.532911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-cef38293-0b74-41f3-8a2a-cf955531c07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:53.548044 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 833/1640] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:53.549745 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863be73-11bf-42de-b65b-6f66178815d2 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:53.560037 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863be73-11bf-42de-b65b-6f66178815d2 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:53.564472 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6367fc-87f6-4c62-a140-4e2ea7a8f0c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:53.565605 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6367fc-87f6-4c62-a140-4e2ea7a8f0c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:53.565988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6367fc-87f6-4c62-a140-4e2ea7a8f0c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:53.566629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6367fc-87f6-4c62-a140-4e2ea7a8f0c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:53.583937 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6367fc-87f6-4c62-a140-4e2ea7a8f0c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:53.585738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6367fc-87f6-4c62-a140-4e2ea7a8f0c7 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:53.602422 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 834/1641] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:53.650556 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:53.711787 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 808/1642] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:53 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:53.733667 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6367fc-87f6-4c62-a140-4e2ea7a8f0c7 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:53.869083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef38293-0b74-41f3-8a2a-cf955531c07c None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:53.887451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef38293-0b74-41f3-8a2a-cf955531c07c None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:53.896388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c557c-852b-4aa2-8079-b1691b5a6cc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:53.897379 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c557c-852b-4aa2-8079-b1691b5a6cc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:53.897883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c557c-852b-4aa2-8079-b1691b5a6cc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:53.898326 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c557c-852b-4aa2-8079-b1691b5a6cc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:53.917022 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:53.922294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c557c-852b-4aa2-8079-b1691b5a6cc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:53.941216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c557c-852b-4aa2-8079-b1691b5a6cc2 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:53.945762 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6367fc-87f6-4c62-a140-4e2ea7a8f0c7 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:53.956497 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bee587-d6c6-4f5f-b5ee-ba4112783a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:53.959786 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bee587-d6c6-4f5f-b5ee-ba4112783a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:53.961060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bee587-d6c6-4f5f-b5ee-ba4112783a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:53.962719 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bee587-d6c6-4f5f-b5ee-ba4112783a84 admin admin] PATH_INFO: `/v3/users/4ed0920154434d74b84fae18deff3449` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:53.964912 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bee587-d6c6-4f5f-b5ee-ba4112783a84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ed0920154434d74b84fae18deff3449)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:53.968178 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-82bee587-d6c6-4f5f-b5ee-ba4112783a84 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:15:53.968178 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 27 21:15:53.968178 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: Jun 27 21:15:53.970563 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bee587-d6c6-4f5f-b5ee-ba4112783a84 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:53.983350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 809/1643] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:53 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:54.151495 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-82bee587-d6c6-4f5f-b5ee-ba4112783a84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ed0920154434d74b84fae18deff3449'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:54.153712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-82bee587-d6c6-4f5f-b5ee-ba4112783a84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ed0920154434d74b84fae18deff3449'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:54.157549 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-82bee587-d6c6-4f5f-b5ee-ba4112783a84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ed0920154434d74b84fae18deff3449'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:54.243295 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-82bee587-d6c6-4f5f-b5ee-ba4112783a84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ed0920154434d74b84fae18deff3449'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:54.277333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-82bee587-d6c6-4f5f-b5ee-ba4112783a84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ed0920154434d74b84fae18deff3449'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:54.318737 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 835/1644] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:15:53 2022] DELETE /identity/v3/users/4ed0920154434d74b84fae18deff3449 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:54.323932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fab854-6a82-4481-9f53-824de3146d67 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:54.324235 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fab854-6a82-4481-9f53-824de3146d67 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:54.324517 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fab854-6a82-4481-9f53-824de3146d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:54.347330 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e0fab854-6a82-4481-9f53-824de3146d67 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:54.349965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e0fab854-6a82-4481-9f53-824de3146d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:54.505292 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 810/1645] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:15:54.562767 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bee587-d6c6-4f5f-b5ee-ba4112783a84 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:54.563402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-82bee587-d6c6-4f5f-b5ee-ba4112783a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:54.716689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bee587-d6c6-4f5f-b5ee-ba4112783a84 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:54.729973 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed0e07-9aa8-462a-bd5f-c45cd8aba65b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:54.730951 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed0e07-9aa8-462a-bd5f-c45cd8aba65b service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:54.731446 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed0e07-9aa8-462a-bd5f-c45cd8aba65b service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:54.731850 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed0e07-9aa8-462a-bd5f-c45cd8aba65b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:54.754662 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed0e07-9aa8-462a-bd5f-c45cd8aba65b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:54.756273 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed0e07-9aa8-462a-bd5f-c45cd8aba65b service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:54.782026 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 836/1646] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:54.853306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fab854-6a82-4481-9f53-824de3146d67 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:54.854964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e0fab854-6a82-4481-9f53-824de3146d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:54.955583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed0e07-9aa8-462a-bd5f-c45cd8aba65b service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:54.957145 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e5ed0e07-9aa8-462a-bd5f-c45cd8aba65b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:54.987316 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed0e07-9aa8-462a-bd5f-c45cd8aba65b service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:55.000037 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e798f32-ef37-446c-9a06-d6d0f3c9de11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:55.002137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e798f32-ef37-446c-9a06-d6d0f3c9de11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:55.002981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e798f32-ef37-446c-9a06-d6d0f3c9de11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:55.005289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e798f32-ef37-446c-9a06-d6d0f3c9de11 admin admin] PATH_INFO: `/v3/users/c6aafd1e5fef4e76a6e73879c775fd8d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:55.008416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e798f32-ef37-446c-9a06-d6d0f3c9de11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6aafd1e5fef4e76a6e73879c775fd8d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:55.013144 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e798f32-ef37-446c-9a06-d6d0f3c9de11 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:55.025777 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fab854-6a82-4481-9f53-824de3146d67 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:55.037331 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6179f-7455-4e17-a1a4-531e73c5ebb6 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a28fd5319e674c5d8367a0893425702a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee56e38d3e44e791b9b82137cba69e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:55.038195 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6179f-7455-4e17-a1a4-531e73c5ebb6 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:55.038985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6179f-7455-4e17-a1a4-531e73c5ebb6 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:55.038985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6179f-7455-4e17-a1a4-531e73c5ebb6 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:55.043157 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6179f-7455-4e17-a1a4-531e73c5ebb6 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:15:55.043621 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6179f-7455-4e17-a1a4-531e73c5ebb6 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:55.045477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6179f-7455-4e17-a1a4-531e73c5ebb6 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:55.072514 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 811/1647] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:15:55.076076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6179f-7455-4e17-a1a4-531e73c5ebb6 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:55.117021 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6179f-7455-4e17-a1a4-531e73c5ebb6 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:55.132043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6e798f32-ef37-446c-9a06-d6d0f3c9de11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6aafd1e5fef4e76a6e73879c775fd8d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:55.134653 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6e798f32-ef37-446c-9a06-d6d0f3c9de11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6aafd1e5fef4e76a6e73879c775fd8d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:55.135404 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6e798f32-ef37-446c-9a06-d6d0f3c9de11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6aafd1e5fef4e76a6e73879c775fd8d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:55.137312 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c635a-e1a2-4353-8a1c-2b703e29ae24 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a28fd5319e674c5d8367a0893425702a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee56e38d3e44e791b9b82137cba69e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:55.138007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c635a-e1a2-4353-8a1c-2b703e29ae24 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:55.138301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c635a-e1a2-4353-8a1c-2b703e29ae24 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:55.138588 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c635a-e1a2-4353-8a1c-2b703e29ae24 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:55.139868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c635a-e1a2-4353-8a1c-2b703e29ae24 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:55.142073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c635a-e1a2-4353-8a1c-2b703e29ae24 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:55.165354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6e798f32-ef37-446c-9a06-d6d0f3c9de11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6aafd1e5fef4e76a6e73879c775fd8d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:55.176951 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:55.183223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6e798f32-ef37-446c-9a06-d6d0f3c9de11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6aafd1e5fef4e76a6e73879c775fd8d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:55.192603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 812/1648] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:55 2022] POST /identity/v3/projects => generated 355 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:55.194484 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c635a-e1a2-4353-8a1c-2b703e29ae24 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:55.198344 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-006c635a-e1a2-4353-8a1c-2b703e29ae24 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:55.211875 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 837/1649] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:15:54 2022] DELETE /identity/v3/users/c6aafd1e5fef4e76a6e73879c775fd8d => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:15:55.214361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8910a-f324-40b7-9fee-3fe3dec20975 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:55.214705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8910a-f324-40b7-9fee-3fe3dec20975 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:55.215084 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8910a-f324-40b7-9fee-3fe3dec20975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:55.255571 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-cea8910a-f324-40b7-9fee-3fe3dec20975 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:55.257802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cea8910a-f324-40b7-9fee-3fe3dec20975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:55.304058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:55.354057 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c635a-e1a2-4353-8a1c-2b703e29ae24 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:55.444091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 838/1650] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:55.447036 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51889eea-16b0-4ce0-80d0-ccf31dcec747 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a28fd5319e674c5d8367a0893425702a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee56e38d3e44e791b9b82137cba69e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:55.447771 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51889eea-16b0-4ce0-80d0-ccf31dcec747 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:55.448063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51889eea-16b0-4ce0-80d0-ccf31dcec747 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:55.448346 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51889eea-16b0-4ce0-80d0-ccf31dcec747 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:55.449103 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51889eea-16b0-4ce0-80d0-ccf31dcec747 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:15:55.449416 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51889eea-16b0-4ce0-80d0-ccf31dcec747 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:55.451041 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51889eea-16b0-4ce0-80d0-ccf31dcec747 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:55.466937 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 813/1651] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:15:55.491847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8910a-f324-40b7-9fee-3fe3dec20975 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:55.493539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cea8910a-f324-40b7-9fee-3fe3dec20975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:55.578207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51889eea-16b0-4ce0-80d0-ccf31dcec747 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:55.597320 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51889eea-16b0-4ce0-80d0-ccf31dcec747 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:55.634111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df880b-a77f-4882-8890-8988c8ea9b88 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a28fd5319e674c5d8367a0893425702a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee56e38d3e44e791b9b82137cba69e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:55.636733 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df880b-a77f-4882-8890-8988c8ea9b88 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:55.641787 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df880b-a77f-4882-8890-8988c8ea9b88 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:55.642605 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df880b-a77f-4882-8890-8988c8ea9b88 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] PATH_INFO: `/v3/projects/da56e4e35f05428682def487f5f8fbeb` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:55.650007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df880b-a77f-4882-8890-8988c8ea9b88 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] RBAC: Authorizing `identity:delete_project(project_id=da56e4e35f05428682def487f5f8fbeb)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:55.660923 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: WARNING py.warnings [None req-02df880b-a77f-4882-8890-8988c8ea9b88 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:15:55.660923 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: warnings.warn(msg) Jun 27 21:15:55.660923 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: Jun 27 21:15:55.666133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df880b-a77f-4882-8890-8988c8ea9b88 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:55.749354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8910a-f324-40b7-9fee-3fe3dec20975 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:55.756961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114f99c-7ce6-4724-8152-409014630af0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:55.757926 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114f99c-7ce6-4724-8152-409014630af0 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:55.759011 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114f99c-7ce6-4724-8152-409014630af0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:55.760383 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114f99c-7ce6-4724-8152-409014630af0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:55.808251 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114f99c-7ce6-4724-8152-409014630af0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:55.810910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114f99c-7ce6-4724-8152-409014630af0 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:55.837871 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 839/1652] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:55.853420 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-02df880b-a77f-4882-8890-8988c8ea9b88 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da56e4e35f05428682def487f5f8fbeb', 'request_id': 'req-02df880b-a77f-4882-8890-8988c8ea9b88'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:55.884353 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-02df880b-a77f-4882-8890-8988c8ea9b88 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da56e4e35f05428682def487f5f8fbeb', 'request_id': 'req-02df880b-a77f-4882-8890-8988c8ea9b88'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:55.898538 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-02df880b-a77f-4882-8890-8988c8ea9b88 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da56e4e35f05428682def487f5f8fbeb', 'request_id': 'req-02df880b-a77f-4882-8890-8988c8ea9b88'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:55.931927 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-02df880b-a77f-4882-8890-8988c8ea9b88 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] The token cache is being invalidate because project da56e4e35f05428682def487f5f8fbeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:15:55.932358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-02df880b-a77f-4882-8890-8988c8ea9b88 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:55.933327 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-02df880b-a77f-4882-8890-8988c8ea9b88 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:55.976232 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 814/1653] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:15:55 2022] DELETE /identity/v3/projects/da56e4e35f05428682def487f5f8fbeb => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:56.364016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114f99c-7ce6-4724-8152-409014630af0 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:56.365691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e114f99c-7ce6-4724-8152-409014630af0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:56.400543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114f99c-7ce6-4724-8152-409014630af0 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:56.463013 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df880b-a77f-4882-8890-8988c8ea9b88 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:56.464654 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-02df880b-a77f-4882-8890-8988c8ea9b88 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:56.508949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa09fe-1b53-4919-988b-b797fa5f70cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:56.510014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa09fe-1b53-4919-988b-b797fa5f70cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:56.510849 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa09fe-1b53-4919-988b-b797fa5f70cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:56.511196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa09fe-1b53-4919-988b-b797fa5f70cc admin admin] PATH_INFO: `/v3/users/f56292d1ba174a02b15853d61f340d57` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:56.513151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa09fe-1b53-4919-988b-b797fa5f70cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f56292d1ba174a02b15853d61f340d57)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:56.516147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa09fe-1b53-4919-988b-b797fa5f70cc admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:56.572463 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:56.594466 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fcfa09fe-1b53-4919-988b-b797fa5f70cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f56292d1ba174a02b15853d61f340d57'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:56.595771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fcfa09fe-1b53-4919-988b-b797fa5f70cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f56292d1ba174a02b15853d61f340d57'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:56.597237 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fcfa09fe-1b53-4919-988b-b797fa5f70cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f56292d1ba174a02b15853d61f340d57'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:56.622890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fcfa09fe-1b53-4919-988b-b797fa5f70cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f56292d1ba174a02b15853d61f340d57'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:56.633581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fcfa09fe-1b53-4919-988b-b797fa5f70cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f56292d1ba174a02b15853d61f340d57'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:56.652596 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 840/1654] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:15:56 2022] DELETE /identity/v3/users/f56292d1ba174a02b15853d61f340d57 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:56.657001 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04848c67-06de-419d-9140-597e5f7d98b0 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:56.657508 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04848c67-06de-419d-9140-597e5f7d98b0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:56.657781 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04848c67-06de-419d-9140-597e5f7d98b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:56.658577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df880b-a77f-4882-8890-8988c8ea9b88 tempest-VolumeQuotaClassesTest-1845600288 tempest-VolumeQuotaClassesTest-1845600288-project-admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:56.687505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-04848c67-06de-419d-9140-597e5f7d98b0 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:56.689434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-04848c67-06de-419d-9140-597e5f7d98b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:56.694062 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:56.836718 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2ca53-bacd-4018-970f-6ac43361fcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:56.838757 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2ca53-bacd-4018-970f-6ac43361fcdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:56.839159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2ca53-bacd-4018-970f-6ac43361fcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:56.839541 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2ca53-bacd-4018-970f-6ac43361fcdb admin admin] PATH_INFO: `/v3/projects/36fbf32e85d84d888f91e99337b3518b` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:56.841490 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea2ca53-bacd-4018-970f-6ac43361fcdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=36fbf32e85d84d888f91e99337b3518b)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:56.854588 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea2ca53-bacd-4018-970f-6ac43361fcdb admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:56.913076 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 841/1655] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:15:56.935167 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04848c67-06de-419d-9140-597e5f7d98b0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:56.936649 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-04848c67-06de-419d-9140-597e5f7d98b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:56.944089 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5ea2ca53-bacd-4018-970f-6ac43361fcdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36fbf32e85d84d888f91e99337b3518b', 'request_id': 'req-5ea2ca53-bacd-4018-970f-6ac43361fcdb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:56.963349 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5ea2ca53-bacd-4018-970f-6ac43361fcdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36fbf32e85d84d888f91e99337b3518b', 'request_id': 'req-5ea2ca53-bacd-4018-970f-6ac43361fcdb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:56.973790 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5ea2ca53-bacd-4018-970f-6ac43361fcdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36fbf32e85d84d888f91e99337b3518b', 'request_id': 'req-5ea2ca53-bacd-4018-970f-6ac43361fcdb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:57.010854 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5ea2ca53-bacd-4018-970f-6ac43361fcdb admin admin] The token cache is being invalidate because project 36fbf32e85d84d888f91e99337b3518b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:15:57.011370 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5ea2ca53-bacd-4018-970f-6ac43361fcdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:57.017251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5ea2ca53-bacd-4018-970f-6ac43361fcdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:57.024815 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 815/1656] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:15:56 2022] DELETE /identity/v3/projects/36fbf32e85d84d888f91e99337b3518b => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:57.028096 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54840b8-e713-42ef-8929-eeacda4f77d0 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:57.030752 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54840b8-e713-42ef-8929-eeacda4f77d0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:57.031050 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54840b8-e713-42ef-8929-eeacda4f77d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:57.057983 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04848c67-06de-419d-9140-597e5f7d98b0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:57.071349 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d54840b8-e713-42ef-8929-eeacda4f77d0 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:57.074009 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d54840b8-e713-42ef-8929-eeacda4f77d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:57.226760 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b6536-ce1b-4d3b-b055-f942f82518ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:57.227819 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b6536-ce1b-4d3b-b055-f942f82518ba service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:57.227819 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b6536-ce1b-4d3b-b055-f942f82518ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:57.228121 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b6536-ce1b-4d3b-b055-f942f82518ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:57.233807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-443b6536-ce1b-4d3b-b055-f942f82518ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:57.247747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 816/1657] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:57.254495 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a348a-f748-4fa3-9b45-b6efc2f38d6b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:57.255280 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a348a-f748-4fa3-9b45-b6efc2f38d6b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:57.255714 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a348a-f748-4fa3-9b45-b6efc2f38d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:57.261216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b6536-ce1b-4d3b-b055-f942f82518ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:57.265425 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b6536-ce1b-4d3b-b055-f942f82518ba service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:57.300393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 842/1658] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:57.414754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a40a348a-f748-4fa3-9b45-b6efc2f38d6b None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:57.428676 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a40a348a-f748-4fa3-9b45-b6efc2f38d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:57.486527 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b6536-ce1b-4d3b-b055-f942f82518ba service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:57.489299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 817/1659] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:57.502480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b6536-ce1b-4d3b-b055-f942f82518ba service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:57.524038 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011be7b-564d-4b08-bd91-b38d89cf67e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:57.525145 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011be7b-564d-4b08-bd91-b38d89cf67e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:57.525545 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011be7b-564d-4b08-bd91-b38d89cf67e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:57.525918 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011be7b-564d-4b08-bd91-b38d89cf67e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:57.526913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011be7b-564d-4b08-bd91-b38d89cf67e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:15:57.527378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011be7b-564d-4b08-bd91-b38d89cf67e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:57.529862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011be7b-564d-4b08-bd91-b38d89cf67e9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:57.545815 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:57.585121 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 843/1660] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:15:57.586481 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011be7b-564d-4b08-bd91-b38d89cf67e9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:57.587208 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a348a-f748-4fa3-9b45-b6efc2f38d6b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:57.590316 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a40a348a-f748-4fa3-9b45-b6efc2f38d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:57.607653 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011be7b-564d-4b08-bd91-b38d89cf67e9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:57.619038 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a348a-f748-4fa3-9b45-b6efc2f38d6b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:57.628991 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefe567-75d9-4653-ad71-9d07e3fc77a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:57.630812 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7500c-774b-460b-af23-44a9d519f4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:57.630906 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefe567-75d9-4653-ad71-9d07e3fc77a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:57.630987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7500c-774b-460b-af23-44a9d519f4f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:57.630987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7500c-774b-460b-af23-44a9d519f4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:57.633341 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefe567-75d9-4653-ad71-9d07e3fc77a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:57.633341 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefe567-75d9-4653-ad71-9d07e3fc77a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:57.633660 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7500c-774b-460b-af23-44a9d519f4f9 admin admin] PATH_INFO: `/v3/projects/2de03c4861a6487d9465abdaa4f3bc53` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:57.633735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cefe567-75d9-4653-ad71-9d07e3fc77a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:57.635779 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7500c-774b-460b-af23-44a9d519f4f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2de03c4861a6487d9465abdaa4f3bc53)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:57.637900 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7500c-774b-460b-af23-44a9d519f4f9 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:57.638021 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cefe567-75d9-4653-ad71-9d07e3fc77a7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:57.701196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 844/1661] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:57 2022] POST /identity/v3/projects => generated 363 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:57.705449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefe567-75d9-4653-ad71-9d07e3fc77a7 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:57.715710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2cefe567-75d9-4653-ad71-9d07e3fc77a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:57.840511 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:57.908208 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eeb7500c-774b-460b-af23-44a9d519f4f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2de03c4861a6487d9465abdaa4f3bc53', 'request_id': 'req-eeb7500c-774b-460b-af23-44a9d519f4f9'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:57.939779 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eeb7500c-774b-460b-af23-44a9d519f4f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2de03c4861a6487d9465abdaa4f3bc53', 'request_id': 'req-eeb7500c-774b-460b-af23-44a9d519f4f9'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:57.955254 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eeb7500c-774b-460b-af23-44a9d519f4f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2de03c4861a6487d9465abdaa4f3bc53', 'request_id': 'req-eeb7500c-774b-460b-af23-44a9d519f4f9'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:57.985464 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefe567-75d9-4653-ad71-9d07e3fc77a7 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:58.014871 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eeb7500c-774b-460b-af23-44a9d519f4f9 admin admin] The token cache is being invalidate because project 2de03c4861a6487d9465abdaa4f3bc53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:15:58.022183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eeb7500c-774b-460b-af23-44a9d519f4f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:58.024494 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eeb7500c-774b-460b-af23-44a9d519f4f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:58.034284 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 818/1662] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:15:57 2022] DELETE /identity/v3/projects/2de03c4861a6487d9465abdaa4f3bc53 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:58.037398 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7500c-774b-460b-af23-44a9d519f4f9 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:58.054958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-eeb7500c-774b-460b-af23-44a9d519f4f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:58.250179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c9823-f257-4d45-99df-d74ba929eae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:58.254151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c9823-f257-4d45-99df-d74ba929eae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:58.254745 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c9823-f257-4d45-99df-d74ba929eae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:58.255237 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c9823-f257-4d45-99df-d74ba929eae1 admin admin] PATH_INFO: `/v3/users/0dcfc6a4bdfa474a8aebfa593b466684` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:58.261461 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c9823-f257-4d45-99df-d74ba929eae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dcfc6a4bdfa474a8aebfa593b466684)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:58.272324 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c9823-f257-4d45-99df-d74ba929eae1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:58.400146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7500c-774b-460b-af23-44a9d519f4f9 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:58.420070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e6591-fbc3-4424-b9b4-db38a4fa4377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:58.420884 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e6591-fbc3-4424-b9b4-db38a4fa4377 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:58.421232 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e6591-fbc3-4424-b9b4-db38a4fa4377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:58.421577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e6591-fbc3-4424-b9b4-db38a4fa4377 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:58.422421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e6591-fbc3-4424-b9b4-db38a4fa4377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:58.423023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1c9c9823-f257-4d45-99df-d74ba929eae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dcfc6a4bdfa474a8aebfa593b466684'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:58.423925 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1c9c9823-f257-4d45-99df-d74ba929eae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dcfc6a4bdfa474a8aebfa593b466684'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:58.424420 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e6591-fbc3-4424-b9b4-db38a4fa4377 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:58.425059 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1c9c9823-f257-4d45-99df-d74ba929eae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dcfc6a4bdfa474a8aebfa593b466684'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:58.450922 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1c9c9823-f257-4d45-99df-d74ba929eae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dcfc6a4bdfa474a8aebfa593b466684'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:58.487400 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1c9c9823-f257-4d45-99df-d74ba929eae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dcfc6a4bdfa474a8aebfa593b466684'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:58.495880 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 819/1663] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:58 2022] POST /identity/v3/users => generated 282 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:58.498310 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e6591-fbc3-4424-b9b4-db38a4fa4377 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:58.500704 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e71e6591-fbc3-4424-b9b4-db38a4fa4377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:58.515652 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:58.522602 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e6591-fbc3-4424-b9b4-db38a4fa4377 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:58.525543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 845/1664] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:15:57 2022] DELETE /identity/v3/users/0dcfc6a4bdfa474a8aebfa593b466684 => generated 0 bytes in 820 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:58.528686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c9823-f257-4d45-99df-d74ba929eae1 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:58.533810 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1c9c9823-f257-4d45-99df-d74ba929eae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:58.669182 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c9823-f257-4d45-99df-d74ba929eae1 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:58.679117 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4647f8-b924-4f33-9645-af287a8d5f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:58.681789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4647f8-b924-4f33-9645-af287a8d5f50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:58.682931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4647f8-b924-4f33-9645-af287a8d5f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:58.684272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4647f8-b924-4f33-9645-af287a8d5f50 admin admin] PATH_INFO: `/v3/users/c8e2121683a44bc288197ef2b668ef0b` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:58.684571 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36da07-128f-488b-bca1-f2a3628095ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:58.686398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36da07-128f-488b-bca1-f2a3628095ed admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:58.686398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36da07-128f-488b-bca1-f2a3628095ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:58.688068 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36da07-128f-488b-bca1-f2a3628095ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:58.688503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4647f8-b924-4f33-9645-af287a8d5f50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8e2121683a44bc288197ef2b668ef0b)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:58.690940 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36da07-128f-488b-bca1-f2a3628095ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:58.694514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36da07-128f-488b-bca1-f2a3628095ed admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:58.694987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4647f8-b924-4f33-9645-af287a8d5f50 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:58.717132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 846/1665] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:58.724797 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41c660-8393-4b4c-8337-4073772ccfc4 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:58.727071 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41c660-8393-4b4c-8337-4073772ccfc4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:58.730974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41c660-8393-4b4c-8337-4073772ccfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:58.771166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7b41c660-8393-4b4c-8337-4073772ccfc4 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:58.773533 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7b41c660-8393-4b4c-8337-4073772ccfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:58.796314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 847/1666] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:58.800246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41c660-8393-4b4c-8337-4073772ccfc4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:58.837113 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41c660-8393-4b4c-8337-4073772ccfc4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:58.848888 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc1c3cc-afe2-419c-b5ba-c24e225607f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:58.851996 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1c3cc-afe2-419c-b5ba-c24e225607f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:58.870044 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1c3cc-afe2-419c-b5ba-c24e225607f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:58.872963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1c3cc-afe2-419c-b5ba-c24e225607f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:58.874439 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc1c3cc-afe2-419c-b5ba-c24e225607f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:58.880073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-cc4647f8-b924-4f33-9645-af287a8d5f50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8e2121683a44bc288197ef2b668ef0b'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:58.881538 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-cc4647f8-b924-4f33-9645-af287a8d5f50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8e2121683a44bc288197ef2b668ef0b'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:58.883178 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc1c3cc-afe2-419c-b5ba-c24e225607f1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:58.883821 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-cc4647f8-b924-4f33-9645-af287a8d5f50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8e2121683a44bc288197ef2b668ef0b'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:58.897288 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=13,keystone:SELECT=83,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:58.908208 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 848/1667] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:58.913028 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc1c3cc-afe2-419c-b5ba-c24e225607f1 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:58.916588 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3fc1c3cc-afe2-419c-b5ba-c24e225607f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:58.923649 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-cc4647f8-b924-4f33-9645-af287a8d5f50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8e2121683a44bc288197ef2b668ef0b'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:58.935656 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-cc4647f8-b924-4f33-9645-af287a8d5f50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8e2121683a44bc288197ef2b668ef0b'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:58.983785 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 820/1668] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:15:58 2022] DELETE /identity/v3/users/c8e2121683a44bc288197ef2b668ef0b => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:58.984695 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4647f8-b924-4f33-9645-af287a8d5f50 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:58.986249 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-cc4647f8-b924-4f33-9645-af287a8d5f50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:59.040943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc1c3cc-afe2-419c-b5ba-c24e225607f1 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:59.106433 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4647f8-b924-4f33-9645-af287a8d5f50 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:59.112431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee85859-5b6a-478c-83cd-fb94c5ae2733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:59.113520 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d7fe2-6eba-44e3-b073-a4413d300750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:59.114043 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee85859-5b6a-478c-83cd-fb94c5ae2733 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:59.114174 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d7fe2-6eba-44e3-b073-a4413d300750 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:59.114456 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d7fe2-6eba-44e3-b073-a4413d300750 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:59.114660 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee85859-5b6a-478c-83cd-fb94c5ae2733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:59.114770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d7fe2-6eba-44e3-b073-a4413d300750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:59.115115 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee85859-5b6a-478c-83cd-fb94c5ae2733 admin admin] PATH_INFO: `/v3/projects/751bf4e00af24e6580fa6491d7472379/users/3730161020a144e0839d8bbe0ad9b6c2/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:59.127215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7f7d7fe2-6eba-44e3-b073-a4413d300750 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:59.134813 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee85859-5b6a-478c-83cd-fb94c5ae2733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=751bf4e00af24e6580fa6491d7472379, user_id=3730161020a144e0839d8bbe0ad9b6c2, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:59.147856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d7fe2-6eba-44e3-b073-a4413d300750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:59.149592 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d7fe2-6eba-44e3-b073-a4413d300750 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:59.156782 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee85859-5b6a-478c-83cd-fb94c5ae2733 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:59.165725 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 849/1669] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:59.166991 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d7fe2-6eba-44e3-b073-a4413d300750 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:59.169548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7f7d7fe2-6eba-44e3-b073-a4413d300750 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:59.189443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d7fe2-6eba-44e3-b073-a4413d300750 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:59.196642 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 821/1670] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:58 2022] PUT /identity/v3/projects/751bf4e00af24e6580fa6491d7472379/users/3730161020a144e0839d8bbe0ad9b6c2/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:59.201273 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2326e84-c488-47d8-9ba6-c6bb12f1e53f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:59.201626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2326e84-c488-47d8-9ba6-c6bb12f1e53f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:59.201914 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2326e84-c488-47d8-9ba6-c6bb12f1e53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:59.248334 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:59.295188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40480b-6b3e-4245-a970-9330571a3b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:59.296825 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40480b-6b3e-4245-a970-9330571a3b0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:59.297331 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40480b-6b3e-4245-a970-9330571a3b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:59.297851 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40480b-6b3e-4245-a970-9330571a3b0d admin admin] PATH_INFO: `/v3/projects/a60658a4e6084eba8bff08bef89a8b8d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:59.301188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40480b-6b3e-4245-a970-9330571a3b0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a60658a4e6084eba8bff08bef89a8b8d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:59.305072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40480b-6b3e-4245-a970-9330571a3b0d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:59.335933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-c2326e84-c488-47d8-9ba6-c6bb12f1e53f None None] MFA Rules not processed for user `14307670704943768a05d7d6e5a7318b`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:59.355711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c2326e84-c488-47d8-9ba6-c6bb12f1e53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:59.412234 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-af40480b-6b3e-4245-a970-9330571a3b0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a60658a4e6084eba8bff08bef89a8b8d', 'request_id': 'req-af40480b-6b3e-4245-a970-9330571a3b0d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:59.427195 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-af40480b-6b3e-4245-a970-9330571a3b0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a60658a4e6084eba8bff08bef89a8b8d', 'request_id': 'req-af40480b-6b3e-4245-a970-9330571a3b0d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:59.434892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-af40480b-6b3e-4245-a970-9330571a3b0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a60658a4e6084eba8bff08bef89a8b8d', 'request_id': 'req-af40480b-6b3e-4245-a970-9330571a3b0d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:59.465598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-af40480b-6b3e-4245-a970-9330571a3b0d admin admin] The token cache is being invalidate because project a60658a4e6084eba8bff08bef89a8b8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:15:59.483504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-af40480b-6b3e-4245-a970-9330571a3b0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:59.485629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-af40480b-6b3e-4245-a970-9330571a3b0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:59.496507 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 850/1671] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:15:59 2022] DELETE /identity/v3/projects/a60658a4e6084eba8bff08bef89a8b8d => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:59.518809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2730587-c181-4832-9ff8-e3038672e5b0 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:59.519464 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2730587-c181-4832-9ff8-e3038672e5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:59.523409 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2730587-c181-4832-9ff8-e3038672e5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:59.639870 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 822/1672] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 442 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:59.643851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2326e84-c488-47d8-9ba6-c6bb12f1e53f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:59.650082 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c2326e84-c488-47d8-9ba6-c6bb12f1e53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:59.651569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c2730587-c181-4832-9ff8-e3038672e5b0 None None] MFA Rules not processed for user `3730161020a144e0839d8bbe0ad9b6c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:59.656459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c2730587-c181-4832-9ff8-e3038672e5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:59.720778 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2326e84-c488-47d8-9ba6-c6bb12f1e53f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:59.898280 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:59.922727 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 851/1673] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:59 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:59.927766 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b713b2b-65b8-4d1b-b2e5-003af2204a9d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:59.928073 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b713b2b-65b8-4d1b-b2e5-003af2204a9d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:59.928358 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b713b2b-65b8-4d1b-b2e5-003af2204a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:59.964998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ffb81-c70f-4ffe-89e7-d5b103cd2b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:59.965938 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ffb81-c70f-4ffe-89e7-d5b103cd2b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:59.966250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ffb81-c70f-4ffe-89e7-d5b103cd2b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:59.966530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ffb81-c70f-4ffe-89e7-d5b103cd2b31 admin admin] PATH_INFO: `/v3/users/a28fd5319e674c5d8367a0893425702a` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:59.970162 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ffb81-c70f-4ffe-89e7-d5b103cd2b31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a28fd5319e674c5d8367a0893425702a)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:59.978629 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ffb81-c70f-4ffe-89e7-d5b103cd2b31 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:00.088460 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5b713b2b-65b8-4d1b-b2e5-003af2204a9d None None] MFA Rules not processed for user `14307670704943768a05d7d6e5a7318b`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:00.098306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5b713b2b-65b8-4d1b-b2e5-003af2204a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:00.133075 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d71ffb81-c70f-4ffe-89e7-d5b103cd2b31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a28fd5319e674c5d8367a0893425702a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:00.133740 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d71ffb81-c70f-4ffe-89e7-d5b103cd2b31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a28fd5319e674c5d8367a0893425702a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:00.134324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d71ffb81-c70f-4ffe-89e7-d5b103cd2b31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a28fd5319e674c5d8367a0893425702a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:00.172358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d71ffb81-c70f-4ffe-89e7-d5b103cd2b31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a28fd5319e674c5d8367a0893425702a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:00.195930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d71ffb81-c70f-4ffe-89e7-d5b103cd2b31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a28fd5319e674c5d8367a0893425702a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:00.246946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 823/1674] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:15:59 2022] DELETE /identity/v3/users/a28fd5319e674c5d8367a0893425702a => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:00.250537 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ffb81-c70f-4ffe-89e7-d5b103cd2b31 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:00.259168 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d71ffb81-c70f-4ffe-89e7-d5b103cd2b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:00.414428 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 852/1675] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:15:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 491 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:00.427525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b713b2b-65b8-4d1b-b2e5-003af2204a9d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:00.433098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5b713b2b-65b8-4d1b-b2e5-003af2204a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:00.456950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:00.604046 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ffb81-c70f-4ffe-89e7-d5b103cd2b31 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:00.630561 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9281872-1dfc-43e7-bbac-19d64a7fa263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:00.631265 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9281872-1dfc-43e7-bbac-19d64a7fa263 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:00.631544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9281872-1dfc-43e7-bbac-19d64a7fa263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:00.632086 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9281872-1dfc-43e7-bbac-19d64a7fa263 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:00.632939 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9281872-1dfc-43e7-bbac-19d64a7fa263 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:00.637037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9281872-1dfc-43e7-bbac-19d64a7fa263 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:00.722949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 824/1676] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:00 2022] POST /identity/v3/projects => generated 363 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:00.726939 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9281872-1dfc-43e7-bbac-19d64a7fa263 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:00.732865 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b713b2b-65b8-4d1b-b2e5-003af2204a9d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:00.905374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:00.941537 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9281872-1dfc-43e7-bbac-19d64a7fa263 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:00.951303 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7677b4-5bd6-4817-affb-98852d4ec6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:00.951806 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7677b4-5bd6-4817-affb-98852d4ec6dc admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:00.952073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7677b4-5bd6-4817-affb-98852d4ec6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:00.952364 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7677b4-5bd6-4817-affb-98852d4ec6dc admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:00.953059 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7677b4-5bd6-4817-affb-98852d4ec6dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:00.957888 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f545f-0f3f-4ed1-a97b-943e57decb33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:00.958212 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f545f-0f3f-4ed1-a97b-943e57decb33 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:00.958494 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f545f-0f3f-4ed1-a97b-943e57decb33 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:00.958770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f545f-0f3f-4ed1-a97b-943e57decb33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:00.966433 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-695f545f-0f3f-4ed1-a97b-943e57decb33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:00.967925 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7677b4-5bd6-4817-affb-98852d4ec6dc admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:01.022189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 825/1677] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:00 2022] POST /identity/v3/users => generated 288 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:01.023162 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7677b4-5bd6-4817-affb-98852d4ec6dc admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:01.026468 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0a7677b4-5bd6-4817-affb-98852d4ec6dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:01.068443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7677b4-5bd6-4817-affb-98852d4ec6dc admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:01.089321 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45493294-bd0b-4ac6-81a7-f09c27d06355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:01.090159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45493294-bd0b-4ac6-81a7-f09c27d06355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:01.090449 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45493294-bd0b-4ac6-81a7-f09c27d06355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:01.090735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45493294-bd0b-4ac6-81a7-f09c27d06355 admin admin] PATH_INFO: `/v3/projects/70cc2f2f62b04ffabb6d5d341ea80244` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:01.095516 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45493294-bd0b-4ac6-81a7-f09c27d06355 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70cc2f2f62b04ffabb6d5d341ea80244)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:01.101459 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45493294-bd0b-4ac6-81a7-f09c27d06355 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:01.121223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f545f-0f3f-4ed1-a97b-943e57decb33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:01.122921 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f545f-0f3f-4ed1-a97b-943e57decb33 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:01.144290 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 853/1678] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 715 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:01.145351 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f545f-0f3f-4ed1-a97b-943e57decb33 service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:01.148614 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-695f545f-0f3f-4ed1-a97b-943e57decb33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:01.176183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f545f-0f3f-4ed1-a97b-943e57decb33 service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:01.190139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615ba14-14dd-4ba2-b52f-244ad21d5ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:01.191056 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615ba14-14dd-4ba2-b52f-244ad21d5ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:01.191343 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615ba14-14dd-4ba2-b52f-244ad21d5ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:01.191635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615ba14-14dd-4ba2-b52f-244ad21d5ed8 admin admin] PATH_INFO: `/v3/projects/474443a05dd94540931e136c5e5abc54` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:01.196052 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615ba14-14dd-4ba2-b52f-244ad21d5ed8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=474443a05dd94540931e136c5e5abc54)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:01.199501 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615ba14-14dd-4ba2-b52f-244ad21d5ed8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:01.275014 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-45493294-bd0b-4ac6-81a7-f09c27d06355 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70cc2f2f62b04ffabb6d5d341ea80244', 'request_id': 'req-45493294-bd0b-4ac6-81a7-f09c27d06355'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:01.303882 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-45493294-bd0b-4ac6-81a7-f09c27d06355 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70cc2f2f62b04ffabb6d5d341ea80244', 'request_id': 'req-45493294-bd0b-4ac6-81a7-f09c27d06355'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:01.307999 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:01.315250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-45493294-bd0b-4ac6-81a7-f09c27d06355 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70cc2f2f62b04ffabb6d5d341ea80244', 'request_id': 'req-45493294-bd0b-4ac6-81a7-f09c27d06355'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:01.344594 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-45493294-bd0b-4ac6-81a7-f09c27d06355 admin admin] The token cache is being invalidate because project 70cc2f2f62b04ffabb6d5d341ea80244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:01.344879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-45493294-bd0b-4ac6-81a7-f09c27d06355 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:01.345575 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-45493294-bd0b-4ac6-81a7-f09c27d06355 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:01.351197 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 826/1679] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:01 2022] DELETE /identity/v3/projects/70cc2f2f62b04ffabb6d5d341ea80244 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:01.351197 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45493294-bd0b-4ac6-81a7-f09c27d06355 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:01.351197 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-45493294-bd0b-4ac6-81a7-f09c27d06355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:01.360565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7615ba14-14dd-4ba2-b52f-244ad21d5ed8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '474443a05dd94540931e136c5e5abc54', 'request_id': 'req-7615ba14-14dd-4ba2-b52f-244ad21d5ed8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:01.392900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7615ba14-14dd-4ba2-b52f-244ad21d5ed8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '474443a05dd94540931e136c5e5abc54', 'request_id': 'req-7615ba14-14dd-4ba2-b52f-244ad21d5ed8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:01.419463 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7615ba14-14dd-4ba2-b52f-244ad21d5ed8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '474443a05dd94540931e136c5e5abc54', 'request_id': 'req-7615ba14-14dd-4ba2-b52f-244ad21d5ed8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:01.453938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7615ba14-14dd-4ba2-b52f-244ad21d5ed8 admin admin] The token cache is being invalidate because project 474443a05dd94540931e136c5e5abc54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:01.454292 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7615ba14-14dd-4ba2-b52f-244ad21d5ed8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:01.454918 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7615ba14-14dd-4ba2-b52f-244ad21d5ed8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:01.462692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 854/1680] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:01 2022] DELETE /identity/v3/projects/474443a05dd94540931e136c5e5abc54 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:01.464482 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef115b06-5532-4e97-829a-f5796f7279d9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:01.464593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef115b06-5532-4e97-829a-f5796f7279d9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:01.464840 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef115b06-5532-4e97-829a-f5796f7279d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:01.519816 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ef115b06-5532-4e97-829a-f5796f7279d9 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:01.523394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ef115b06-5532-4e97-829a-f5796f7279d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:01.567640 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45493294-bd0b-4ac6-81a7-f09c27d06355 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:01.641444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 855/1681] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:01.643213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98897168-f1e9-44cc-9b1a-e1c71718d055 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:01.643512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98897168-f1e9-44cc-9b1a-e1c71718d055 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:01.643813 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98897168-f1e9-44cc-9b1a-e1c71718d055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:01.671954 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-98897168-f1e9-44cc-9b1a-e1c71718d055 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:01.673388 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-98897168-f1e9-44cc-9b1a-e1c71718d055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:01.689482 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 856/1682] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:01.693564 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788911f-50f9-4104-ae7b-b7210b96163a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:01.694032 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788911f-50f9-4104-ae7b-b7210b96163a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:01.694325 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788911f-50f9-4104-ae7b-b7210b96163a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:01.724470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-1788911f-50f9-4104-ae7b-b7210b96163a None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:01.725731 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1788911f-50f9-4104-ae7b-b7210b96163a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:01.738378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 857/1683] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:01.739773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9c6b2-74a0-4195-955c-42cd0f30652e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:01.740217 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9c6b2-74a0-4195-955c-42cd0f30652e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:01.740657 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9c6b2-74a0-4195-955c-42cd0f30652e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:01.761654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-97a9c6b2-74a0-4195-955c-42cd0f30652e None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:01.762921 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-97a9c6b2-74a0-4195-955c-42cd0f30652e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:01.764388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791eb9a3-7837-4092-9f3d-24838bbc46e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:01.767377 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791eb9a3-7837-4092-9f3d-24838bbc46e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:01.767998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791eb9a3-7837-4092-9f3d-24838bbc46e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:01.768445 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791eb9a3-7837-4092-9f3d-24838bbc46e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:01.771995 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791eb9a3-7837-4092-9f3d-24838bbc46e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:01.775207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791eb9a3-7837-4092-9f3d-24838bbc46e5 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:01.781502 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 858/1684] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:01.783426 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9c6b2-74a0-4195-955c-42cd0f30652e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:01.805722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9c6b2-74a0-4195-955c-42cd0f30652e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:01.809052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 827/1685] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 460 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:01.811730 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791eb9a3-7837-4092-9f3d-24838bbc46e5 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:01.823384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d153b09-4481-467a-8d7b-70a8062d8a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:01.824213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d153b09-4481-467a-8d7b-70a8062d8a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:01.824575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d153b09-4481-467a-8d7b-70a8062d8a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:01.824925 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d153b09-4481-467a-8d7b-70a8062d8a12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:01.825761 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d153b09-4481-467a-8d7b-70a8062d8a12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:01.826148 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d153b09-4481-467a-8d7b-70a8062d8a12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:01.828132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d153b09-4481-467a-8d7b-70a8062d8a12 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:01.832114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:01.843292 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 859/1686] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:16:01.844123 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d153b09-4481-467a-8d7b-70a8062d8a12 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:01.845993 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8d153b09-4481-467a-8d7b-70a8062d8a12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:01.846464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791eb9a3-7837-4092-9f3d-24838bbc46e5 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:01.861921 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a912bf3-5275-4f58-8057-a565d8fc174a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:01.864737 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a912bf3-5275-4f58-8057-a565d8fc174a admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:01.866029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a912bf3-5275-4f58-8057-a565d8fc174a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:01.866481 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a912bf3-5275-4f58-8057-a565d8fc174a admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:01.869467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d153b09-4481-467a-8d7b-70a8062d8a12 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:01.870375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a912bf3-5275-4f58-8057-a565d8fc174a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:01.872129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a912bf3-5275-4f58-8057-a565d8fc174a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:01.876141 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a912bf3-5275-4f58-8057-a565d8fc174a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:01.876381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c849113-03e1-490e-a1a4-4cc8d0ba7fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:01.877489 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c849113-03e1-490e-a1a4-4cc8d0ba7fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:01.878129 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c849113-03e1-490e-a1a4-4cc8d0ba7fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:01.878584 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c849113-03e1-490e-a1a4-4cc8d0ba7fd5 admin admin] PATH_INFO: `/v3/projects/3588b2dd33eb432a8390c762aa843167/users/6fcd34aa21224d69a174432fd190110e/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:01.902356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c849113-03e1-490e-a1a4-4cc8d0ba7fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3588b2dd33eb432a8390c762aa843167, user_id=6fcd34aa21224d69a174432fd190110e, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:01.907846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c849113-03e1-490e-a1a4-4cc8d0ba7fd5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:01.917477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 828/1687] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:16:01.920383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a912bf3-5275-4f58-8057-a565d8fc174a admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:01.936340 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 860/1688] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:01 2022] PUT /identity/v3/projects/3588b2dd33eb432a8390c762aa843167/users/6fcd34aa21224d69a174432fd190110e/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:01.938634 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c849113-03e1-490e-a1a4-4cc8d0ba7fd5 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:01.955143 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a912bf3-5275-4f58-8057-a565d8fc174a admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:02.033832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c849113-03e1-490e-a1a4-4cc8d0ba7fd5 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:02.046984 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae372bf-47b0-4953-b11f-286484940b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:02.047910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae372bf-47b0-4953-b11f-286484940b28 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:02.048300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae372bf-47b0-4953-b11f-286484940b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:02.049627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae372bf-47b0-4953-b11f-286484940b28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:02.050954 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae372bf-47b0-4953-b11f-286484940b28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:02.053631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae372bf-47b0-4953-b11f-286484940b28 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:02.068751 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:02.126286 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 861/1689] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:01 2022] POST /identity/v3/projects => generated 375 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:02.128389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae372bf-47b0-4953-b11f-286484940b28 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:02.141920 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e28359-6516-46de-82de-25d6d5f86bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:02.142626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e28359-6516-46de-82de-25d6d5f86bcb admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:02.142910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e28359-6516-46de-82de-25d6d5f86bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:02.143193 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e28359-6516-46de-82de-25d6d5f86bcb admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:02.144314 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e28359-6516-46de-82de-25d6d5f86bcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:02.146352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e28359-6516-46de-82de-25d6d5f86bcb admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:02.216113 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 829/1690] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:01 2022] POST /identity/v3/projects => generated 387 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:02.217756 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e28359-6516-46de-82de-25d6d5f86bcb admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:02.274534 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae372bf-47b0-4953-b11f-286484940b28 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:02.380109 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e28359-6516-46de-82de-25d6d5f86bcb admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:02.391138 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbc4c5-790a-4ac7-8e79-41fa7fdad5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:02.391657 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbc4c5-790a-4ac7-8e79-41fa7fdad5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:02.391948 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbc4c5-790a-4ac7-8e79-41fa7fdad5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:02.392224 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbc4c5-790a-4ac7-8e79-41fa7fdad5d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:02.393087 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbc4c5-790a-4ac7-8e79-41fa7fdad5d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:02.395393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbc4c5-790a-4ac7-8e79-41fa7fdad5d1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:02.396575 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475665e-4f12-4bef-9d73-1150aab60707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:02.397288 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475665e-4f12-4bef-9d73-1150aab60707 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:02.397573 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475665e-4f12-4bef-9d73-1150aab60707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:02.397856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475665e-4f12-4bef-9d73-1150aab60707 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:02.398573 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475665e-4f12-4bef-9d73-1150aab60707 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:02.400600 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475665e-4f12-4bef-9d73-1150aab60707 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:02.415412 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 862/1691] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:02.415412 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbc4c5-790a-4ac7-8e79-41fa7fdad5d1 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:02.437798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbc4c5-790a-4ac7-8e79-41fa7fdad5d1 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:02.441451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 830/1692] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:02 2022] POST /identity/v3/users => generated 288 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:02.441451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475665e-4f12-4bef-9d73-1150aab60707 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:02.453686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e99b6-3530-4bb3-ae8a-9db80e9450c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:02.454063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e99b6-3530-4bb3-ae8a-9db80e9450c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:02.454337 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e99b6-3530-4bb3-ae8a-9db80e9450c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:02.455022 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e99b6-3530-4bb3-ae8a-9db80e9450c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:02.455318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e99b6-3530-4bb3-ae8a-9db80e9450c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:02.460761 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475665e-4f12-4bef-9d73-1150aab60707 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:02.461778 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e99b6-3530-4bb3-ae8a-9db80e9450c5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:02.474067 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a74ac-a657-4538-a4d8-b6b1f9ff813c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:02.475014 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a74ac-a657-4538-a4d8-b6b1f9ff813c admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:02.475366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a74ac-a657-4538-a4d8-b6b1f9ff813c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:02.475674 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a74ac-a657-4538-a4d8-b6b1f9ff813c admin admin] PATH_INFO: `/v3/projects/3588b2dd33eb432a8390c762aa843167/users/6fcd34aa21224d69a174432fd190110e/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:02.480108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a74ac-a657-4538-a4d8-b6b1f9ff813c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3588b2dd33eb432a8390c762aa843167, user_id=6fcd34aa21224d69a174432fd190110e, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:02.487525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a74ac-a657-4538-a4d8-b6b1f9ff813c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:02.507717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 863/1693] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:02 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:02.508988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e99b6-3530-4bb3-ae8a-9db80e9450c5 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:02.523669 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 831/1694] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:02 2022] PUT /identity/v3/projects/3588b2dd33eb432a8390c762aa843167/users/6fcd34aa21224d69a174432fd190110e/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:02.523669 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a74ac-a657-4538-a4d8-b6b1f9ff813c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:02.533133 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e99b6-3530-4bb3-ae8a-9db80e9450c5 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:02.629718 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:02.665700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a74ac-a657-4538-a4d8-b6b1f9ff813c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:02.679068 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875f5ef-80ae-4405-ab0d-c8511fee53e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:02.679886 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875f5ef-80ae-4405-ab0d-c8511fee53e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:02.680893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875f5ef-80ae-4405-ab0d-c8511fee53e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:02.681529 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875f5ef-80ae-4405-ab0d-c8511fee53e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:02.683782 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c875f5ef-80ae-4405-ab0d-c8511fee53e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:02.687259 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c875f5ef-80ae-4405-ab0d-c8511fee53e6 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:02.687371 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93bc4a-523c-4bf1-bcbe-e3a11d589dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:02.688119 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93bc4a-523c-4bf1-bcbe-e3a11d589dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:02.688400 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93bc4a-523c-4bf1-bcbe-e3a11d589dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:02.688675 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93bc4a-523c-4bf1-bcbe-e3a11d589dd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:02.689418 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93bc4a-523c-4bf1-bcbe-e3a11d589dd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:02.691051 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93bc4a-523c-4bf1-bcbe-e3a11d589dd0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:02.716908 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 864/1695] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:02.718266 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 832/1696] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:02.719333 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875f5ef-80ae-4405-ab0d-c8511fee53e6 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:02.725973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c875f5ef-80ae-4405-ab0d-c8511fee53e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:02.733904 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645703b-025c-4e50-b871-97a360ab8024 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:02.734297 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645703b-025c-4e50-b871-97a360ab8024 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:02.734635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645703b-025c-4e50-b871-97a360ab8024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:02.755041 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875f5ef-80ae-4405-ab0d-c8511fee53e6 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:02.772349 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42d366-7134-44ae-8841-ab6a7e4572db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:02.773765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42d366-7134-44ae-8841-ab6a7e4572db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:02.774148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42d366-7134-44ae-8841-ab6a7e4572db admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:02.774664 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42d366-7134-44ae-8841-ab6a7e4572db admin admin] PATH_INFO: `/v3/projects/c1ee56e38d3e44e791b9b82137cba69e` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:02.779429 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42d366-7134-44ae-8841-ab6a7e4572db admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1ee56e38d3e44e791b9b82137cba69e)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:02.782565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42d366-7134-44ae-8841-ab6a7e4572db admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:02.790085 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:02.803974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e645703b-025c-4e50-b871-97a360ab8024 None None] MFA Rules not processed for user `6fcd34aa21224d69a174432fd190110e`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:02.807501 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e645703b-025c-4e50-b871-97a360ab8024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:02.893076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-0c42d366-7134-44ae-8841-ab6a7e4572db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1ee56e38d3e44e791b9b82137cba69e', 'request_id': 'req-0c42d366-7134-44ae-8841-ab6a7e4572db'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:02.907301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-0c42d366-7134-44ae-8841-ab6a7e4572db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1ee56e38d3e44e791b9b82137cba69e', 'request_id': 'req-0c42d366-7134-44ae-8841-ab6a7e4572db'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:02.924967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-0c42d366-7134-44ae-8841-ab6a7e4572db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1ee56e38d3e44e791b9b82137cba69e', 'request_id': 'req-0c42d366-7134-44ae-8841-ab6a7e4572db'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:02.942220 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-0c42d366-7134-44ae-8841-ab6a7e4572db admin admin] The token cache is being invalidate because project c1ee56e38d3e44e791b9b82137cba69e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:02.944569 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-0c42d366-7134-44ae-8841-ab6a7e4572db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:02.945425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-0c42d366-7134-44ae-8841-ab6a7e4572db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:02.949995 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 833/1697] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:02 2022] DELETE /identity/v3/projects/c1ee56e38d3e44e791b9b82137cba69e => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:02.953568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42d366-7134-44ae-8841-ab6a7e4572db admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:02.954886 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0c42d366-7134-44ae-8841-ab6a7e4572db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:03.000870 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 865/1698] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:02 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:03.002046 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645703b-025c-4e50-b871-97a360ab8024 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:03.018554 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645703b-025c-4e50-b871-97a360ab8024 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:03.144806 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42d366-7134-44ae-8841-ab6a7e4572db admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:03.158931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7b288-e8a5-4db8-ba3f-68ebdd027c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:03.160918 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7b288-e8a5-4db8-ba3f-68ebdd027c98 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:03.161200 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7b288-e8a5-4db8-ba3f-68ebdd027c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:03.161521 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7b288-e8a5-4db8-ba3f-68ebdd027c98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:03.162467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7b288-e8a5-4db8-ba3f-68ebdd027c98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:03.166337 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10f956-f056-4139-8bfc-5de01b524753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:03.167216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10f956-f056-4139-8bfc-5de01b524753 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:03.168210 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10f956-f056-4139-8bfc-5de01b524753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:03.168495 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10f956-f056-4139-8bfc-5de01b524753 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:03.169335 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7b288-e8a5-4db8-ba3f-68ebdd027c98 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:03.171315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10f956-f056-4139-8bfc-5de01b524753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:03.176831 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10f956-f056-4139-8bfc-5de01b524753 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:03.203069 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 834/1699] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:03.203069 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70fb74-4f17-45e9-a194-ccde02e80d73 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:03.203069 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70fb74-4f17-45e9-a194-ccde02e80d73 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:03.203069 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70fb74-4f17-45e9-a194-ccde02e80d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:03.210038 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 866/1700] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:03.211297 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10f956-f056-4139-8bfc-5de01b524753 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:03.214252 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2f10f956-f056-4139-8bfc-5de01b524753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:03.256880 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5d70fb74-4f17-45e9-a194-ccde02e80d73 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:03.261974 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5d70fb74-4f17-45e9-a194-ccde02e80d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:03.279982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 835/1701] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:03.287080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70fb74-4f17-45e9-a194-ccde02e80d73 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:03.301324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70fb74-4f17-45e9-a194-ccde02e80d73 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:03.312467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52076dd-7258-4c80-baf5-8306fba13052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:03.313787 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52076dd-7258-4c80-baf5-8306fba13052 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:03.313973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52076dd-7258-4c80-baf5-8306fba13052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:03.314296 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52076dd-7258-4c80-baf5-8306fba13052 admin admin] PATH_INFO: `/v3/projects/32b7a2210a464addbb01905f27b1db48/users/accef9f6fe3b4f7a9459a846998e0594/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:03.349091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10f956-f056-4139-8bfc-5de01b524753 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:03.351677 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52076dd-7258-4c80-baf5-8306fba13052 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32b7a2210a464addbb01905f27b1db48, user_id=accef9f6fe3b4f7a9459a846998e0594, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:03.356242 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52076dd-7258-4c80-baf5-8306fba13052 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:03.363437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a3107-a83f-43f4-b97a-118566d6d40f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:03.366956 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a3107-a83f-43f4-b97a-118566d6d40f service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:03.367384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a3107-a83f-43f4-b97a-118566d6d40f service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:03.367678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a3107-a83f-43f4-b97a-118566d6d40f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:03.371554 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-340a3107-a83f-43f4-b97a-118566d6d40f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:03.393012 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 836/1702] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:03 2022] PUT /identity/v3/projects/32b7a2210a464addbb01905f27b1db48/users/accef9f6fe3b4f7a9459a846998e0594/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:03.397431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52076dd-7258-4c80-baf5-8306fba13052 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:03.400916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c52076dd-7258-4c80-baf5-8306fba13052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:03.498361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:03.501957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:03.553279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a3107-a83f-43f4-b97a-118566d6d40f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:03.553623 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a3107-a83f-43f4-b97a-118566d6d40f service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:03.574553 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52076dd-7258-4c80-baf5-8306fba13052 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:03.579103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 867/1703] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:03 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:03.584464 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb67cda-6ca2-4057-8cdd-8c5e403d7712 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:03.584750 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb67cda-6ca2-4057-8cdd-8c5e403d7712 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:03.585024 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb67cda-6ca2-4057-8cdd-8c5e403d7712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:03.587057 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacab43b-f61c-4645-8386-403ec16280f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:03.588010 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacab43b-f61c-4645-8386-403ec16280f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:03.588286 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacab43b-f61c-4645-8386-403ec16280f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:03.588553 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacab43b-f61c-4645-8386-403ec16280f3 admin admin] PATH_INFO: `/v3/projects/fbd72b1384ec4fccaef7b8dbb2418a0d/users/f64baf6ff7be4ea38ed5b7b6fdd05cc7/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:03.636037 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-bbb67cda-6ca2-4057-8cdd-8c5e403d7712 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:03.637194 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bbb67cda-6ca2-4057-8cdd-8c5e403d7712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:03.642388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacab43b-f61c-4645-8386-403ec16280f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbd72b1384ec4fccaef7b8dbb2418a0d, user_id=f64baf6ff7be4ea38ed5b7b6fdd05cc7, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:03.646044 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacab43b-f61c-4645-8386-403ec16280f3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:03.669398 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 868/1704] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:03.671303 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1dd99-8bd9-430d-b28b-67177cb773b7 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:03.671303 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1dd99-8bd9-430d-b28b-67177cb773b7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:03.671303 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1dd99-8bd9-430d-b28b-67177cb773b7 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:03.672974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 869/1705] 10.222.0.27 () {54 vars in 912 bytes} [Mon Jun 27 21:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:16:03.673988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de3c87-ac38-4548-9dc2-8658b5cab34a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:03.674227 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de3c87-ac38-4548-9dc2-8658b5cab34a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:03.674521 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de3c87-ac38-4548-9dc2-8658b5cab34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:03.680934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 837/1706] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:03 2022] PUT /identity/v3/projects/fbd72b1384ec4fccaef7b8dbb2418a0d/users/f64baf6ff7be4ea38ed5b7b6fdd05cc7/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:03.682179 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacab43b-f61c-4645-8386-403ec16280f3 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:03.761988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b3de3c87-ac38-4548-9dc2-8658b5cab34a None None] MFA Rules not processed for user `accef9f6fe3b4f7a9459a846998e0594`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:03.763351 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b3de3c87-ac38-4548-9dc2-8658b5cab34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:03.887360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacab43b-f61c-4645-8386-403ec16280f3 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:03.893733 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 870/1707] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:03.898037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666aecfa-9739-4ff3-8fe4-5fff645f53f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:03.898599 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff17af2-79de-40c0-b5bc-25376e893585 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:03.899220 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff17af2-79de-40c0-b5bc-25376e893585 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:03.899766 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff17af2-79de-40c0-b5bc-25376e893585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:03.902658 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666aecfa-9739-4ff3-8fe4-5fff645f53f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:03.902876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666aecfa-9739-4ff3-8fe4-5fff645f53f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:03.903163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666aecfa-9739-4ff3-8fe4-5fff645f53f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:03.904616 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666aecfa-9739-4ff3-8fe4-5fff645f53f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:03.904945 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666aecfa-9739-4ff3-8fe4-5fff645f53f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:03.918487 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666aecfa-9739-4ff3-8fe4-5fff645f53f7 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:03.958874 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9ff17af2-79de-40c0-b5bc-25376e893585 None None] MFA Rules not processed for user `9c7ce29db6b14176a4c9be436bcddc6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:03.960390 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9ff17af2-79de-40c0-b5bc-25376e893585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:03.971983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 838/1708] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:16:03.973654 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcc822-6c9c-4b7a-9844-d7b71b081b21 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:03.977836 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcc822-6c9c-4b7a-9844-d7b71b081b21 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:03.978333 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcc822-6c9c-4b7a-9844-d7b71b081b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:04.070960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-3bbcc822-6c9c-4b7a-9844-d7b71b081b21 None None] MFA Rules not processed for user `f64baf6ff7be4ea38ed5b7b6fdd05cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:04.075358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3bbcc822-6c9c-4b7a-9844-d7b71b081b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:04.100538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 871/1709] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Jun 27 21:16:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:04.107412 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae7540-63db-4b0f-b70e-d5e42b1ba30c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:04.107735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae7540-63db-4b0f-b70e-d5e42b1ba30c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:04.108016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae7540-63db-4b0f-b70e-d5e42b1ba30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:04.189410 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-19ae7540-63db-4b0f-b70e-d5e42b1ba30c None None] MFA Rules not processed for user `74b2295521bc45509ef6b68c1885ac6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:04.190737 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-19ae7540-63db-4b0f-b70e-d5e42b1ba30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:04.292432 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 839/1710] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:04.293426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcc822-6c9c-4b7a-9844-d7b71b081b21 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:04.309950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:04.321523 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcc822-6c9c-4b7a-9844-d7b71b081b21 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:04.328930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5967a7-7f88-40a0-b595-4fdc2cdbf5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:04.330359 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5967a7-7f88-40a0-b595-4fdc2cdbf5a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:04.332097 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5967a7-7f88-40a0-b595-4fdc2cdbf5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:04.332097 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5967a7-7f88-40a0-b595-4fdc2cdbf5a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:04.332556 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5967a7-7f88-40a0-b595-4fdc2cdbf5a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:04.335996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5967a7-7f88-40a0-b595-4fdc2cdbf5a1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:04.341923 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 872/1711] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:04.342800 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae7540-63db-4b0f-b70e-d5e42b1ba30c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:04.379838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae7540-63db-4b0f-b70e-d5e42b1ba30c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:04.391256 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab18e5d-7c42-4050-a328-93108b8d36d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:04.392406 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab18e5d-7c42-4050-a328-93108b8d36d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:04.392406 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab18e5d-7c42-4050-a328-93108b8d36d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:04.393037 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab18e5d-7c42-4050-a328-93108b8d36d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:04.393737 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab18e5d-7c42-4050-a328-93108b8d36d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:04.401814 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab18e5d-7c42-4050-a328-93108b8d36d3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:04.416067 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 840/1712] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:04 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:04.422139 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5967a7-7f88-40a0-b595-4fdc2cdbf5a1 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:04.427064 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3e5967a7-7f88-40a0-b595-4fdc2cdbf5a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:04.469275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 873/1713] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:04 2022] POST /identity/v3/projects => generated 409 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:04.470004 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab18e5d-7c42-4050-a328-93108b8d36d3 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:04.511534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:04.583479 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5967a7-7f88-40a0-b595-4fdc2cdbf5a1 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:04.627647 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab18e5d-7c42-4050-a328-93108b8d36d3 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:04.634626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0e576-9f7c-48a6-9b72-880f0434b895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:04.635525 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0e576-9f7c-48a6-9b72-880f0434b895 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:04.636258 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0e576-9f7c-48a6-9b72-880f0434b895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:04.636640 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0e576-9f7c-48a6-9b72-880f0434b895 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:04.638066 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0e576-9f7c-48a6-9b72-880f0434b895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:04.641173 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0e576-9f7c-48a6-9b72-880f0434b895 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:04.669821 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfd19d-6f27-41b2-8167-67081409a3f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:04.671447 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfd19d-6f27-41b2-8167-67081409a3f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:04.671823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfd19d-6f27-41b2-8167-67081409a3f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:04.672262 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfd19d-6f27-41b2-8167-67081409a3f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:04.683663 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cfd19d-6f27-41b2-8167-67081409a3f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:04.685664 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cfd19d-6f27-41b2-8167-67081409a3f6 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:04.696002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 841/1714] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:04.696955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfd19d-6f27-41b2-8167-67081409a3f6 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:04.698508 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-99cfd19d-6f27-41b2-8167-67081409a3f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:04.701580 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 874/1715] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:04 2022] POST /identity/v3/projects => generated 387 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:04.705479 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0e576-9f7c-48a6-9b72-880f0434b895 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:04.805618 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0e576-9f7c-48a6-9b72-880f0434b895 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:04.811443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ceb9fe-699f-4347-8dc7-2d15b2e03c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:04.812328 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ceb9fe-699f-4347-8dc7-2d15b2e03c88 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:04.813079 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ceb9fe-699f-4347-8dc7-2d15b2e03c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:04.813452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ceb9fe-699f-4347-8dc7-2d15b2e03c88 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:04.814557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ceb9fe-699f-4347-8dc7-2d15b2e03c88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:04.816723 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ceb9fe-699f-4347-8dc7-2d15b2e03c88 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:04.833997 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfd19d-6f27-41b2-8167-67081409a3f6 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:04.855836 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 875/1716] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:04 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:04.856745 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ceb9fe-699f-4347-8dc7-2d15b2e03c88 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:04.875596 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ceb9fe-699f-4347-8dc7-2d15b2e03c88 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:04.886801 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e9308-d5b9-4d1a-ae1e-6fdc7aad701e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:04.888064 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e9308-d5b9-4d1a-ae1e-6fdc7aad701e admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:04.888998 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e9308-d5b9-4d1a-ae1e-6fdc7aad701e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:04.889469 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e9308-d5b9-4d1a-ae1e-6fdc7aad701e admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:04.890690 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e9308-d5b9-4d1a-ae1e-6fdc7aad701e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:04.893766 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e9308-d5b9-4d1a-ae1e-6fdc7aad701e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:04.957869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 876/1717] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:04 2022] POST /identity/v3/users => generated 305 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:04.960017 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e9308-d5b9-4d1a-ae1e-6fdc7aad701e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:04.962435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-937e9308-d5b9-4d1a-ae1e-6fdc7aad701e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:05.011907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f30743-9f84-4f35-b561-6875219def18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:05.014124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e9308-d5b9-4d1a-ae1e-6fdc7aad701e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:05.015016 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f30743-9f84-4f35-b561-6875219def18 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:05.019481 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f30743-9f84-4f35-b561-6875219def18 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:05.019481 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f30743-9f84-4f35-b561-6875219def18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:05.033934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe34a6e-d2ff-42cb-af20-3013102fce34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:05.037300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe34a6e-d2ff-42cb-af20-3013102fce34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:05.038154 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe34a6e-d2ff-42cb-af20-3013102fce34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:05.038154 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe34a6e-d2ff-42cb-af20-3013102fce34 admin admin] PATH_INFO: `/v3/users/3730161020a144e0839d8bbe0ad9b6c2` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:05.042876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe34a6e-d2ff-42cb-af20-3013102fce34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3730161020a144e0839d8bbe0ad9b6c2)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:05.049289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe34a6e-d2ff-42cb-af20-3013102fce34 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:05.067974 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:05.115796 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:INSERT=8,keystone:DELETE=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:05.190079 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fbe34a6e-d2ff-42cb-af20-3013102fce34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3730161020a144e0839d8bbe0ad9b6c2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:05.191964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fbe34a6e-d2ff-42cb-af20-3013102fce34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3730161020a144e0839d8bbe0ad9b6c2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:05.192776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fbe34a6e-d2ff-42cb-af20-3013102fce34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3730161020a144e0839d8bbe0ad9b6c2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:05.208730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fbe34a6e-d2ff-42cb-af20-3013102fce34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3730161020a144e0839d8bbe0ad9b6c2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:05.220797 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fbe34a6e-d2ff-42cb-af20-3013102fce34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3730161020a144e0839d8bbe0ad9b6c2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:05.256380 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f30743-9f84-4f35-b561-6875219def18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:05.261182 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f30743-9f84-4f35-b561-6875219def18 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:05.264174 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 877/1718] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:04 2022] DELETE /identity/v3/users/3730161020a144e0839d8bbe0ad9b6c2 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:05.265478 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe34a6e-d2ff-42cb-af20-3013102fce34 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:05.284579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fbe34a6e-d2ff-42cb-af20-3013102fce34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:05.288315 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 842/1719] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:05.295105 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f30743-9f84-4f35-b561-6875219def18 service nova] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:05.309952 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f30743-9f84-4f35-b561-6875219def18 service nova] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:05.389059 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18807b52-ad2e-4514-b555-b3f9993197f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:05.389826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18807b52-ad2e-4514-b555-b3f9993197f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:05.390208 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18807b52-ad2e-4514-b555-b3f9993197f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:05.390497 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18807b52-ad2e-4514-b555-b3f9993197f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:05.391852 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18807b52-ad2e-4514-b555-b3f9993197f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:05.393530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18807b52-ad2e-4514-b555-b3f9993197f3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:05.419203 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 843/1720] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:05.420213 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18807b52-ad2e-4514-b555-b3f9993197f3 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:05.428820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-18807b52-ad2e-4514-b555-b3f9993197f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:05.435985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe34a6e-d2ff-42cb-af20-3013102fce34 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:05.441973 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093fb88-a5d3-48df-93c7-2c6078a34237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:05.442792 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093fb88-a5d3-48df-93c7-2c6078a34237 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:05.443194 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093fb88-a5d3-48df-93c7-2c6078a34237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:05.443710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093fb88-a5d3-48df-93c7-2c6078a34237 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:05.445488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e093fb88-a5d3-48df-93c7-2c6078a34237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:05.447745 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e093fb88-a5d3-48df-93c7-2c6078a34237 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:05.460249 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18807b52-ad2e-4514-b555-b3f9993197f3 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:05.472240 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c32ce-87fe-44e9-9b44-137a23af451b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:05.473580 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c32ce-87fe-44e9-9b44-137a23af451b admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:05.473900 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c32ce-87fe-44e9-9b44-137a23af451b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:05.474401 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c32ce-87fe-44e9-9b44-137a23af451b admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:05.475723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c32ce-87fe-44e9-9b44-137a23af451b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:05.477545 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c32ce-87fe-44e9-9b44-137a23af451b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:05.482016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 878/1721] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:05 2022] POST /identity/v3/users => generated 294 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:05.482016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee4a32-ff60-4595-a7c2-a38a36c373c2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:05.482016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee4a32-ff60-4595-a7c2-a38a36c373c2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:05.482016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee4a32-ff60-4595-a7c2-a38a36c373c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:05.489844 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 844/1722] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:05.490875 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c32ce-87fe-44e9-9b44-137a23af451b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:05.492931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7c1c32ce-87fe-44e9-9b44-137a23af451b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:05.509201 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c32ce-87fe-44e9-9b44-137a23af451b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:05.516485 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ce06a-3e88-4bb7-bc36-edcb484cd2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:05.517452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ce06a-3e88-4bb7-bc36-edcb484cd2a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:05.517748 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ce06a-3e88-4bb7-bc36-edcb484cd2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:05.518032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ce06a-3e88-4bb7-bc36-edcb484cd2a3 admin admin] PATH_INFO: `/v3/users/1831abf2fa2449b0b339511da158ce18` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:05.519699 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ce06a-3e88-4bb7-bc36-edcb484cd2a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1831abf2fa2449b0b339511da158ce18)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:05.521666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ce06a-3e88-4bb7-bc36-edcb484cd2a3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:05.539883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5cee4a32-ff60-4595-a7c2-a38a36c373c2 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:05.541399 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5cee4a32-ff60-4595-a7c2-a38a36c373c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:05.567293 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 879/1723] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:05.567293 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee4a32-ff60-4595-a7c2-a38a36c373c2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:05.568359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5cee4a32-ff60-4595-a7c2-a38a36c373c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:05.626042 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee4a32-ff60-4595-a7c2-a38a36c373c2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:05.645352 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2911a3-fd58-4f45-91e5-a1538ac88015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:05.646086 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2911a3-fd58-4f45-91e5-a1538ac88015 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:05.648134 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2911a3-fd58-4f45-91e5-a1538ac88015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:05.650524 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2911a3-fd58-4f45-91e5-a1538ac88015 admin admin] PATH_INFO: `/v3/projects/9205bb90c58d48d588f164eb67ce3a5b/users/4b5007f27ea843dd821b22e99e737d18/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:05.654866 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-732ce06a-3e88-4bb7-bc36-edcb484cd2a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1831abf2fa2449b0b339511da158ce18'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:05.659713 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-732ce06a-3e88-4bb7-bc36-edcb484cd2a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1831abf2fa2449b0b339511da158ce18'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:05.660464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-732ce06a-3e88-4bb7-bc36-edcb484cd2a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1831abf2fa2449b0b339511da158ce18'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:05.684573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2911a3-fd58-4f45-91e5-a1538ac88015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9205bb90c58d48d588f164eb67ce3a5b, user_id=4b5007f27ea843dd821b22e99e737d18, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:05.689481 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2911a3-fd58-4f45-91e5-a1538ac88015 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:05.690609 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-732ce06a-3e88-4bb7-bc36-edcb484cd2a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1831abf2fa2449b0b339511da158ce18'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:05.717867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-732ce06a-3e88-4bb7-bc36-edcb484cd2a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1831abf2fa2449b0b339511da158ce18'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:05.735030 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 880/1724] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:05 2022] PUT /identity/v3/projects/9205bb90c58d48d588f164eb67ce3a5b/users/4b5007f27ea843dd821b22e99e737d18/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:05.738551 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2911a3-fd58-4f45-91e5-a1538ac88015 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:05.741768 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cd2911a3-fd58-4f45-91e5-a1538ac88015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:05.787970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 845/1725] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:05 2022] DELETE /identity/v3/users/1831abf2fa2449b0b339511da158ce18 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:05.790540 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ce06a-3e88-4bb7-bc36-edcb484cd2a3 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:05.795731 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-732ce06a-3e88-4bb7-bc36-edcb484cd2a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:05.827326 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:05.828665 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:05.834027 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2911a3-fd58-4f45-91e5-a1538ac88015 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:05.954009 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ce06a-3e88-4bb7-bc36-edcb484cd2a3 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:05.965751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335025b-72d8-43f5-94f0-f1b289acd22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:05.967814 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335025b-72d8-43f5-94f0-f1b289acd22a admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:05.968073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335025b-72d8-43f5-94f0-f1b289acd22a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:05.968148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335025b-72d8-43f5-94f0-f1b289acd22a admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:05.970978 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335025b-72d8-43f5-94f0-f1b289acd22a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:05.979407 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335025b-72d8-43f5-94f0-f1b289acd22a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:05.997130 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e1b36-3e3e-41b9-a030-34bd12de0d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:05.997959 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e1b36-3e3e-41b9-a030-34bd12de0d2c admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:05.998223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e1b36-3e3e-41b9-a030-34bd12de0d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:05.999036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e1b36-3e3e-41b9-a030-34bd12de0d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:06.000221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e1b36-3e3e-41b9-a030-34bd12de0d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:06.002207 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e1b36-3e3e-41b9-a030-34bd12de0d2c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:06.010236 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 846/1726] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:06.014242 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335025b-72d8-43f5-94f0-f1b289acd22a admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:06.016199 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4335025b-72d8-43f5-94f0-f1b289acd22a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:06.028299 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 881/1727] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:06.028299 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076e684-8de2-4ebc-afb6-5b1128400e68 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:06.028555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076e684-8de2-4ebc-afb6-5b1128400e68 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:06.028746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076e684-8de2-4ebc-afb6-5b1128400e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:06.072998 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e076e684-8de2-4ebc-afb6-5b1128400e68 None None] MFA Rules not processed for user `30cbde26a1964f22902c24917006fb90`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:06.076516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e076e684-8de2-4ebc-afb6-5b1128400e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:06.114218 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335025b-72d8-43f5-94f0-f1b289acd22a admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:06.151096 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2bc7a-dbd5-4881-b7be-7b61715afe90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:06.151995 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2bc7a-dbd5-4881-b7be-7b61715afe90 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:06.152275 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2bc7a-dbd5-4881-b7be-7b61715afe90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:06.152546 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2bc7a-dbd5-4881-b7be-7b61715afe90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:06.159232 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a8e2bc7a-dbd5-4881-b7be-7b61715afe90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:06.175587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2bc7a-dbd5-4881-b7be-7b61715afe90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:06.178051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2bc7a-dbd5-4881-b7be-7b61715afe90 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:06.194846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 847/1728] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:06.210984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2bc7a-dbd5-4881-b7be-7b61715afe90 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:06.212977 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a8e2bc7a-dbd5-4881-b7be-7b61715afe90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:06.228695 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2bc7a-dbd5-4881-b7be-7b61715afe90 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:06.230632 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 882/1729] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:06.232178 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c698b-85cc-4383-bc73-99f1292eac49 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:06.233140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c698b-85cc-4383-bc73-99f1292eac49 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:06.233433 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c698b-85cc-4383-bc73-99f1292eac49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:06.247742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed8119-734d-45fd-9805-eceeb854a7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:06.248854 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed8119-734d-45fd-9805-eceeb854a7e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:06.249431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed8119-734d-45fd-9805-eceeb854a7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:06.250061 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed8119-734d-45fd-9805-eceeb854a7e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:06.251039 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed8119-734d-45fd-9805-eceeb854a7e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:06.260583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed8119-734d-45fd-9805-eceeb854a7e6 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:06.276847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0b1c698b-85cc-4383-bc73-99f1292eac49 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:06.278115 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0b1c698b-85cc-4383-bc73-99f1292eac49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:06.284553 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 848/1730] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:06.290238 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 883/1731] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:06.290907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89c06b-0294-4eb7-8fca-01e1b76ef571 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:06.291185 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c698b-85cc-4383-bc73-99f1292eac49 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:06.291644 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89c06b-0294-4eb7-8fca-01e1b76ef571 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:06.292115 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89c06b-0294-4eb7-8fca-01e1b76ef571 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:06.301324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 849/1732] 10.222.0.27 () {54 vars in 960 bytes} [Mon Jun 27 21:16:06 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:16:06.303918 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89c06b-0294-4eb7-8fca-01e1b76ef571 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:06.309129 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c698b-85cc-4383-bc73-99f1292eac49 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:06.326515 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5805f0b-83eb-4ead-96fa-3e125e52eb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:06.327716 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5805f0b-83eb-4ead-96fa-3e125e52eb22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:06.328003 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5805f0b-83eb-4ead-96fa-3e125e52eb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:06.328992 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5805f0b-83eb-4ead-96fa-3e125e52eb22 admin admin] PATH_INFO: `/v3/projects/77e08e720d64447fabb7a05eba60d92a/users/9d4b5bdccab6438ab42203c259ec6860/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:06.349738 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89c06b-0294-4eb7-8fca-01e1b76ef571 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:06.363945 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2e71a-a92a-44c1-a403-d4241b780c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:06.365023 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2e71a-a92a-44c1-a403-d4241b780c0f admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:06.365332 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2e71a-a92a-44c1-a403-d4241b780c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:06.365617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2e71a-a92a-44c1-a403-d4241b780c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:06.366440 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2e71a-a92a-44c1-a403-d4241b780c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:06.366828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5805f0b-83eb-4ead-96fa-3e125e52eb22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77e08e720d64447fabb7a05eba60d92a, user_id=9d4b5bdccab6438ab42203c259ec6860, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:06.368151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2e71a-a92a-44c1-a403-d4241b780c0f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:06.372268 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5805f0b-83eb-4ead-96fa-3e125e52eb22 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:06.395828 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 850/1733] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:06.400121 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2e71a-a92a-44c1-a403-d4241b780c0f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:06.402964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-aea2e71a-a92a-44c1-a403-d4241b780c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:06.422671 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 884/1734] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:06 2022] PUT /identity/v3/projects/77e08e720d64447fabb7a05eba60d92a/users/9d4b5bdccab6438ab42203c259ec6860/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:06.424054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5805f0b-83eb-4ead-96fa-3e125e52eb22 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:06.649761 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5805f0b-83eb-4ead-96fa-3e125e52eb22 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:06.661755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43dea0-ebbb-4312-ad30-4184b59d4873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:06.663938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43dea0-ebbb-4312-ad30-4184b59d4873 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:06.664232 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43dea0-ebbb-4312-ad30-4184b59d4873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:06.664516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43dea0-ebbb-4312-ad30-4184b59d4873 admin admin] PATH_INFO: `/v3/projects/9205bb90c58d48d588f164eb67ce3a5b/users/4b5007f27ea843dd821b22e99e737d18/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:06.670777 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43dea0-ebbb-4312-ad30-4184b59d4873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9205bb90c58d48d588f164eb67ce3a5b, user_id=4b5007f27ea843dd821b22e99e737d18, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:06.674752 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2e71a-a92a-44c1-a403-d4241b780c0f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:06.678457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43dea0-ebbb-4312-ad30-4184b59d4873 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:06.684509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:06.708677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 885/1735] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:06 2022] PUT /identity/v3/projects/9205bb90c58d48d588f164eb67ce3a5b/users/4b5007f27ea843dd821b22e99e737d18/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:06.715182 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e55f02-dc62-4df2-af70-b9473c9676f4 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:06.715468 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e55f02-dc62-4df2-af70-b9473c9676f4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:06.716066 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e55f02-dc62-4df2-af70-b9473c9676f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:06.759412 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:06.775953 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-13e55f02-dc62-4df2-af70-b9473c9676f4 None None] MFA Rules not processed for user `1f5fa9b0b0314ee9bb7e16b8464307b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:06.777304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-13e55f02-dc62-4df2-af70-b9473c9676f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:06.875433 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b449c23-2f9c-4839-b9de-79ef7f646ab8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:06.876153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b449c23-2f9c-4839-b9de-79ef7f646ab8 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:06.876439 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b449c23-2f9c-4839-b9de-79ef7f646ab8 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:06.876720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b449c23-2f9c-4839-b9de-79ef7f646ab8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:06.985972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 886/1736] 10.222.0.27 () {58 vars in 1057 bytes} [Mon Jun 27 21:16:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:06.987458 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e55f02-dc62-4df2-af70-b9473c9676f4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:07.001310 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-13e55f02-dc62-4df2-af70-b9473c9676f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:07.072874 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b449c23-2f9c-4839-b9de-79ef7f646ab8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:07.077537 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b449c23-2f9c-4839-b9de-79ef7f646ab8 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:07.093143 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 851/1737] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:07.107401 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b449c23-2f9c-4839-b9de-79ef7f646ab8 service nova] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:07.131123 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b449c23-2f9c-4839-b9de-79ef7f646ab8 service nova] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:07.171598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e55f02-dc62-4df2-af70-b9473c9676f4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:07.181200 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ad62d-f5bf-4588-8e02-cdcba960c65a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:07.182022 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ad62d-f5bf-4588-8e02-cdcba960c65a service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:07.182407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ad62d-f5bf-4588-8e02-cdcba960c65a service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:07.182809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ad62d-f5bf-4588-8e02-cdcba960c65a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:07.343328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248a4fe-27c0-40e4-91b3-ed64424f7018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:07.346711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248a4fe-27c0-40e4-91b3-ed64424f7018 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:07.347098 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248a4fe-27c0-40e4-91b3-ed64424f7018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:07.348567 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248a4fe-27c0-40e4-91b3-ed64424f7018 admin admin] PATH_INFO: `/v3/projects/b99f36b12f8642dbaa1b951102994fcf/users/16ff0377c9e14dda8cd7b5e65cc23986/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:07.385762 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248a4fe-27c0-40e4-91b3-ed64424f7018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b99f36b12f8642dbaa1b951102994fcf, user_id=16ff0377c9e14dda8cd7b5e65cc23986, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:07.394443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248a4fe-27c0-40e4-91b3-ed64424f7018 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:07.421809 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 852/1738] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:07 2022] PUT /identity/v3/projects/b99f36b12f8642dbaa1b951102994fcf/users/16ff0377c9e14dda8cd7b5e65cc23986/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:07.423146 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ad62d-f5bf-4588-8e02-cdcba960c65a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:07.425334 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ad62d-f5bf-4588-8e02-cdcba960c65a service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:07.438800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c5b92-b714-430c-9a73-dc68c3178bdb None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:07.439128 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c5b92-b714-430c-9a73-dc68c3178bdb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:07.441804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c5b92-b714-430c-9a73-dc68c3178bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:07.456645 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 887/1739] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:07.465012 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac4536-d04a-4e8e-b5cd-325c98ce4373 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:07.465362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac4536-d04a-4e8e-b5cd-325c98ce4373 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:07.465706 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac4536-d04a-4e8e-b5cd-325c98ce4373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:07.532843 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-365c5b92-b714-430c-9a73-dc68c3178bdb None None] MFA Rules not processed for user `9d4b5bdccab6438ab42203c259ec6860`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:07.534494 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-365c5b92-b714-430c-9a73-dc68c3178bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:07.591165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d8ac4536-d04a-4e8e-b5cd-325c98ce4373 None None] MFA Rules not processed for user `4b5007f27ea843dd821b22e99e737d18`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:07.592472 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d8ac4536-d04a-4e8e-b5cd-325c98ce4373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:07.618224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:07.687960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:07.703440 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 853/1740] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:07 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:07.712746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c5b92-b714-430c-9a73-dc68c3178bdb None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:07.796201 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 888/1741] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:07 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:07.797279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac4536-d04a-4e8e-b5cd-325c98ce4373 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:07.800707 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d8ac4536-d04a-4e8e-b5cd-325c98ce4373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:07.957430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c5b92-b714-430c-9a73-dc68c3178bdb None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:07.991111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7e5ea-93b9-4d03-9167-78d0308f97d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5fa9b0b0314ee9bb7e16b8464307b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:07.991978 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7e5ea-93b9-4d03-9167-78d0308f97d8 service placement] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:07.992469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7e5ea-93b9-4d03-9167-78d0308f97d8 service placement] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:07.992841 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7e5ea-93b9-4d03-9167-78d0308f97d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:07.998577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-edb7e5ea-93b9-4d03-9167-78d0308f97d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:08.009195 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac4536-d04a-4e8e-b5cd-325c98ce4373 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:08.031499 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84cf9d-a7ab-4aae-8b32-63ff4176f26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:08.032436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84cf9d-a7ab-4aae-8b32-63ff4176f26e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:08.032727 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84cf9d-a7ab-4aae-8b32-63ff4176f26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:08.033005 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84cf9d-a7ab-4aae-8b32-63ff4176f26e admin admin] PATH_INFO: `/v3/projects/751bf4e00af24e6580fa6491d7472379` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:08.036414 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa84cf9d-a7ab-4aae-8b32-63ff4176f26e admin admin] RBAC: Authorizing `identity:delete_project(project_id=751bf4e00af24e6580fa6491d7472379)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:08.037268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7e5ea-93b9-4d03-9167-78d0308f97d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:08.038425 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa84cf9d-a7ab-4aae-8b32-63ff4176f26e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:08.042737 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7e5ea-93b9-4d03-9167-78d0308f97d8 service placement] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:08.101660 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 854/1742] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:08.105967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c0564-bee6-4491-b4b2-92839e6519e5 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:08.106463 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c0564-bee6-4491-b4b2-92839e6519e5 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:08.107175 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c0564-bee6-4491-b4b2-92839e6519e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:08.177302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-aa84cf9d-a7ab-4aae-8b32-63ff4176f26e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '751bf4e00af24e6580fa6491d7472379', 'request_id': 'req-aa84cf9d-a7ab-4aae-8b32-63ff4176f26e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:08.183905 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ae9c0564-bee6-4491-b4b2-92839e6519e5 None None] MFA Rules not processed for user `16ff0377c9e14dda8cd7b5e65cc23986`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:08.187685 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ae9c0564-bee6-4491-b4b2-92839e6519e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:08.200337 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-aa84cf9d-a7ab-4aae-8b32-63ff4176f26e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '751bf4e00af24e6580fa6491d7472379', 'request_id': 'req-aa84cf9d-a7ab-4aae-8b32-63ff4176f26e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:08.208883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-aa84cf9d-a7ab-4aae-8b32-63ff4176f26e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '751bf4e00af24e6580fa6491d7472379', 'request_id': 'req-aa84cf9d-a7ab-4aae-8b32-63ff4176f26e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:08.244922 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-aa84cf9d-a7ab-4aae-8b32-63ff4176f26e admin admin] The token cache is being invalidate because project 751bf4e00af24e6580fa6491d7472379 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:08.245283 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-aa84cf9d-a7ab-4aae-8b32-63ff4176f26e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:08.252092 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-aa84cf9d-a7ab-4aae-8b32-63ff4176f26e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:08.263989 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 889/1743] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:07 2022] DELETE /identity/v3/projects/751bf4e00af24e6580fa6491d7472379 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:08.263989 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84cf9d-a7ab-4aae-8b32-63ff4176f26e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:08.267250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-aa84cf9d-a7ab-4aae-8b32-63ff4176f26e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:08.380080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 855/1744] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:08.380080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c0564-bee6-4491-b4b2-92839e6519e5 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:08.386357 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ae9c0564-bee6-4491-b4b2-92839e6519e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:08.456728 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:08.469279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84cf9d-a7ab-4aae-8b32-63ff4176f26e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:08.492354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cace567-a974-4521-826e-64de78c09d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:08.493050 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cace567-a974-4521-826e-64de78c09d44 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:08.493332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cace567-a974-4521-826e-64de78c09d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:08.493603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cace567-a974-4521-826e-64de78c09d44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:08.494307 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cace567-a974-4521-826e-64de78c09d44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:08.496329 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cace567-a974-4521-826e-64de78c09d44 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:08.566549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c0564-bee6-4491-b4b2-92839e6519e5 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:08.575478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a78b2-3dfa-4d89-b97c-6a11f307e3fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:08.576356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a78b2-3dfa-4d89-b97c-6a11f307e3fa service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:08.576527 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a78b2-3dfa-4d89-b97c-6a11f307e3fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:08.576771 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a78b2-3dfa-4d89-b97c-6a11f307e3fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:08.584319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1b8a78b2-3dfa-4d89-b97c-6a11f307e3fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:08.600116 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:08.603955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 890/1745] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:08 2022] POST /identity/v3/projects => generated 409 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:08.604660 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cace567-a974-4521-826e-64de78c09d44 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:08.622510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7cace567-a974-4521-826e-64de78c09d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:08.720910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a78b2-3dfa-4d89-b97c-6a11f307e3fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:08.726443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a78b2-3dfa-4d89-b97c-6a11f307e3fa service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:08.780259 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 856/1746] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:08.784835 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a78b2-3dfa-4d89-b97c-6a11f307e3fa service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:08.788104 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1b8a78b2-3dfa-4d89-b97c-6a11f307e3fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:08.837579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a78b2-3dfa-4d89-b97c-6a11f307e3fa service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:08.891698 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cace567-a974-4521-826e-64de78c09d44 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:08.906466 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5ebd3-23e0-4969-865b-09ce85e98017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:08.907480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5ebd3-23e0-4969-865b-09ce85e98017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:08.907932 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5ebd3-23e0-4969-865b-09ce85e98017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:08.908296 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5ebd3-23e0-4969-865b-09ce85e98017 admin admin] PATH_INFO: `/v3/users/6fcd34aa21224d69a174432fd190110e` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:08.911559 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5ebd3-23e0-4969-865b-09ce85e98017 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fcd34aa21224d69a174432fd190110e)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:08.916100 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5ebd3-23e0-4969-865b-09ce85e98017 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:09.018524 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9d10f-80de-4b71-960d-1e9cf9ce611c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:09.023849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9d10f-80de-4b71-960d-1e9cf9ce611c admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:09.024038 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9d10f-80de-4b71-960d-1e9cf9ce611c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:09.024303 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9d10f-80de-4b71-960d-1e9cf9ce611c admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:09.025052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9d10f-80de-4b71-960d-1e9cf9ce611c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:09.029339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9d10f-80de-4b71-960d-1e9cf9ce611c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:09.089063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d4d5ebd3-23e0-4969-865b-09ce85e98017 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fcd34aa21224d69a174432fd190110e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:09.091019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 857/1747] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:08 2022] POST /identity/v3/projects => generated 387 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:09.091498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d4d5ebd3-23e0-4969-865b-09ce85e98017 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fcd34aa21224d69a174432fd190110e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:09.092064 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9d10f-80de-4b71-960d-1e9cf9ce611c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:09.094604 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d4d5ebd3-23e0-4969-865b-09ce85e98017 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fcd34aa21224d69a174432fd190110e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:09.096780 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d6e9d10f-80de-4b71-960d-1e9cf9ce611c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:09.129055 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d4d5ebd3-23e0-4969-865b-09ce85e98017 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fcd34aa21224d69a174432fd190110e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:09.160130 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d4d5ebd3-23e0-4969-865b-09ce85e98017 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fcd34aa21224d69a174432fd190110e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:09.193788 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 891/1748] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:08 2022] DELETE /identity/v3/users/6fcd34aa21224d69a174432fd190110e => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:09.193788 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5ebd3-23e0-4969-865b-09ce85e98017 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:09.196940 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d4d5ebd3-23e0-4969-865b-09ce85e98017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:09.360872 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9d10f-80de-4b71-960d-1e9cf9ce611c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:09.384315 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbaaef1-f7e2-431e-a44d-2ab1098eee3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:09.385031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbaaef1-f7e2-431e-a44d-2ab1098eee3f admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:09.385312 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbaaef1-f7e2-431e-a44d-2ab1098eee3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:09.385588 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbaaef1-f7e2-431e-a44d-2ab1098eee3f admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:09.386300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbaaef1-f7e2-431e-a44d-2ab1098eee3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:09.388297 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbaaef1-f7e2-431e-a44d-2ab1098eee3f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:09.400436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5ebd3-23e0-4969-865b-09ce85e98017 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:09.418894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3ca90-eda6-4ea2-900e-2ca7662d6bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:09.421574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3ca90-eda6-4ea2-900e-2ca7662d6bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:09.422052 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3ca90-eda6-4ea2-900e-2ca7662d6bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:09.423632 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3ca90-eda6-4ea2-900e-2ca7662d6bf9 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:09.425020 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3ca90-eda6-4ea2-900e-2ca7662d6bf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:09.430267 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3ca90-eda6-4ea2-900e-2ca7662d6bf9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:09.461439 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:09.466586 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 858/1749] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:09 2022] POST /identity/v3/users => generated 311 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:09.467247 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbaaef1-f7e2-431e-a44d-2ab1098eee3f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:09.496615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3dbaaef1-f7e2-431e-a44d-2ab1098eee3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:09.532143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 892/1750] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:09 2022] POST /identity/v3/users => generated 300 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:09.535481 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3ca90-eda6-4ea2-900e-2ca7662d6bf9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:09.548404 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbaaef1-f7e2-431e-a44d-2ab1098eee3f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:09.565664 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d0d1b-6fa9-48b9-9e70-5c5d9642ba8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:09.578752 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d0d1b-6fa9-48b9-9e70-5c5d9642ba8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:09.578752 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d0d1b-6fa9-48b9-9e70-5c5d9642ba8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:09.578752 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d0d1b-6fa9-48b9-9e70-5c5d9642ba8d admin admin] PATH_INFO: `/v3/projects/1e275005751843ed8390794673be7bf9` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:09.578935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3ca90-eda6-4ea2-900e-2ca7662d6bf9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:09.579003 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d0d1b-6fa9-48b9-9e70-5c5d9642ba8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e275005751843ed8390794673be7bf9)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:09.582812 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d0d1b-6fa9-48b9-9e70-5c5d9642ba8d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:09.611584 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da5a4b-d8f4-44f0-8267-b86db77f0591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:09.613826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da5a4b-d8f4-44f0-8267-b86db77f0591 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:09.614130 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da5a4b-d8f4-44f0-8267-b86db77f0591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:09.614414 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da5a4b-d8f4-44f0-8267-b86db77f0591 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:09.617370 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da5a4b-d8f4-44f0-8267-b86db77f0591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:09.622099 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da5a4b-d8f4-44f0-8267-b86db77f0591 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:09.656838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 893/1751] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:09.658663 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da5a4b-d8f4-44f0-8267-b86db77f0591 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:09.666909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:09.693171 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da5a4b-d8f4-44f0-8267-b86db77f0591 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:09.726244 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f70d0d1b-6fa9-48b9-9e70-5c5d9642ba8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e275005751843ed8390794673be7bf9', 'request_id': 'req-f70d0d1b-6fa9-48b9-9e70-5c5d9642ba8d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:09.750044 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f70d0d1b-6fa9-48b9-9e70-5c5d9642ba8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e275005751843ed8390794673be7bf9', 'request_id': 'req-f70d0d1b-6fa9-48b9-9e70-5c5d9642ba8d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:09.758589 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f70d0d1b-6fa9-48b9-9e70-5c5d9642ba8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e275005751843ed8390794673be7bf9', 'request_id': 'req-f70d0d1b-6fa9-48b9-9e70-5c5d9642ba8d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:09.785717 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f70d0d1b-6fa9-48b9-9e70-5c5d9642ba8d admin admin] The token cache is being invalidate because project 1e275005751843ed8390794673be7bf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:09.785994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f70d0d1b-6fa9-48b9-9e70-5c5d9642ba8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:09.786967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f70d0d1b-6fa9-48b9-9e70-5c5d9642ba8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:09.789408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 859/1752] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:09 2022] DELETE /identity/v3/projects/1e275005751843ed8390794673be7bf9 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:09.790227 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d0d1b-6fa9-48b9-9e70-5c5d9642ba8d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:09.792297 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f70d0d1b-6fa9-48b9-9e70-5c5d9642ba8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:09.878954 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894612e-5050-416c-9a45-ef29a3b363b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:09.880193 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894612e-5050-416c-9a45-ef29a3b363b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:09.880537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894612e-5050-416c-9a45-ef29a3b363b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:09.881250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894612e-5050-416c-9a45-ef29a3b363b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:09.882125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8894612e-5050-416c-9a45-ef29a3b363b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:09.884521 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8894612e-5050-416c-9a45-ef29a3b363b7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:09.900916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 894/1753] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:09.902583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48052fd1-30fe-4f92-a927-880576ab31b2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:09.902964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48052fd1-30fe-4f92-a927-880576ab31b2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:09.903330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48052fd1-30fe-4f92-a927-880576ab31b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:09.943679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-48052fd1-30fe-4f92-a927-880576ab31b2 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:09.945301 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-48052fd1-30fe-4f92-a927-880576ab31b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:09.966905 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 895/1754] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:09.966905 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48052fd1-30fe-4f92-a927-880576ab31b2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:09.966905 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-48052fd1-30fe-4f92-a927-880576ab31b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:10.012846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48052fd1-30fe-4f92-a927-880576ab31b2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:10.019719 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338561dc-90d9-4ab0-b1d3-cb0b042bf054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:10.020689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338561dc-90d9-4ab0-b1d3-cb0b042bf054 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:10.021004 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338561dc-90d9-4ab0-b1d3-cb0b042bf054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:10.021283 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338561dc-90d9-4ab0-b1d3-cb0b042bf054 admin admin] PATH_INFO: `/v3/projects/ea95f83fd4114cacbc2db2d90ffc7333/users/3fa9ea54b4f84903b6f86ff7aee3d698/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:10.038300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d0d1b-6fa9-48b9-9e70-5c5d9642ba8d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:10.060419 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338561dc-90d9-4ab0-b1d3-cb0b042bf054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea95f83fd4114cacbc2db2d90ffc7333, user_id=3fa9ea54b4f84903b6f86ff7aee3d698, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:10.066710 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6ffb0-29b6-4eeb-88d2-5cd5fa376a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:10.069801 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338561dc-90d9-4ab0-b1d3-cb0b042bf054 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:10.070298 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6ffb0-29b6-4eeb-88d2-5cd5fa376a78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:10.070679 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6ffb0-29b6-4eeb-88d2-5cd5fa376a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:10.071197 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6ffb0-29b6-4eeb-88d2-5cd5fa376a78 admin admin] PATH_INFO: `/v3/projects/46e7a6bb901246db8f10ec329b108d9f/users/ac1e17641ae0443396bb42a84762f4a6/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:10.106769 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 896/1755] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:09 2022] PUT /identity/v3/projects/ea95f83fd4114cacbc2db2d90ffc7333/users/3fa9ea54b4f84903b6f86ff7aee3d698/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:10.108132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11a6ca-6063-4c2e-8f52-6e1f0a02511f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:10.108423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11a6ca-6063-4c2e-8f52-6e1f0a02511f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:10.108706 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11a6ca-6063-4c2e-8f52-6e1f0a02511f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:10.120439 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6ffb0-29b6-4eeb-88d2-5cd5fa376a78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46e7a6bb901246db8f10ec329b108d9f, user_id=ac1e17641ae0443396bb42a84762f4a6, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:10.124585 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6ffb0-29b6-4eeb-88d2-5cd5fa376a78 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:10.128495 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8f11a6ca-6063-4c2e-8f52-6e1f0a02511f None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:10.129654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8f11a6ca-6063-4c2e-8f52-6e1f0a02511f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:10.177728 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 860/1756] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:09 2022] PUT /identity/v3/projects/46e7a6bb901246db8f10ec329b108d9f/users/ac1e17641ae0443396bb42a84762f4a6/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:10.179086 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6ffb0-29b6-4eeb-88d2-5cd5fa376a78 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:10.252474 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 897/1757] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:10.253757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11a6ca-6063-4c2e-8f52-6e1f0a02511f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:10.265257 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11a6ca-6063-4c2e-8f52-6e1f0a02511f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:10.330252 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6ffb0-29b6-4eeb-88d2-5cd5fa376a78 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:10.348135 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531153f0-0875-43fa-93a5-54ca1cdcb4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:10.348871 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531153f0-0875-43fa-93a5-54ca1cdcb4e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:10.349135 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531153f0-0875-43fa-93a5-54ca1cdcb4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:10.349410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531153f0-0875-43fa-93a5-54ca1cdcb4e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:10.350296 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531153f0-0875-43fa-93a5-54ca1cdcb4e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:10.361930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531153f0-0875-43fa-93a5-54ca1cdcb4e6 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:10.373148 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:10.386363 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 861/1758] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:10.394940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531153f0-0875-43fa-93a5-54ca1cdcb4e6 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:10.413902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531153f0-0875-43fa-93a5-54ca1cdcb4e6 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:10.414633 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783a236-9f87-4064-bd0e-39681ec847a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:10.415330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783a236-9f87-4064-bd0e-39681ec847a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:10.415632 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783a236-9f87-4064-bd0e-39681ec847a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:10.415922 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783a236-9f87-4064-bd0e-39681ec847a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:10.416677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783a236-9f87-4064-bd0e-39681ec847a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:10.418659 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783a236-9f87-4064-bd0e-39681ec847a9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:10.432859 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87778390-8ff3-4a21-9b42-be276c59079b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:10.432859 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87778390-8ff3-4a21-9b42-be276c59079b admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:10.432859 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87778390-8ff3-4a21-9b42-be276c59079b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:10.432859 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87778390-8ff3-4a21-9b42-be276c59079b admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:10.433214 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 898/1759] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:10.433874 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783a236-9f87-4064-bd0e-39681ec847a9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:10.435023 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87778390-8ff3-4a21-9b42-be276c59079b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:10.435336 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87778390-8ff3-4a21-9b42-be276c59079b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:10.437119 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87778390-8ff3-4a21-9b42-be276c59079b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:10.441034 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0783a236-9f87-4064-bd0e-39681ec847a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:10.465107 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 862/1760] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:16:10.466075 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87778390-8ff3-4a21-9b42-be276c59079b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:10.468938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783a236-9f87-4064-bd0e-39681ec847a9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:10.480672 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f57a14-b475-4675-94cf-a161d1e4fc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:10.481587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f57a14-b475-4675-94cf-a161d1e4fc7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:10.482269 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f57a14-b475-4675-94cf-a161d1e4fc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:10.482557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f57a14-b475-4675-94cf-a161d1e4fc7b admin admin] PATH_INFO: `/v3/projects/3588b2dd33eb432a8390c762aa843167` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:10.484495 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f57a14-b475-4675-94cf-a161d1e4fc7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3588b2dd33eb432a8390c762aa843167)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:10.486472 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f57a14-b475-4675-94cf-a161d1e4fc7b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:10.492789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87778390-8ff3-4a21-9b42-be276c59079b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:10.498274 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef18f44-a692-451f-9d3c-57e2739eeb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:10.499272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef18f44-a692-451f-9d3c-57e2739eeb0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:10.500374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef18f44-a692-451f-9d3c-57e2739eeb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:10.500801 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef18f44-a692-451f-9d3c-57e2739eeb0d admin admin] PATH_INFO: `/v3/projects/ea95f83fd4114cacbc2db2d90ffc7333/users/3fa9ea54b4f84903b6f86ff7aee3d698/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:10.503453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef18f44-a692-451f-9d3c-57e2739eeb0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea95f83fd4114cacbc2db2d90ffc7333, user_id=3fa9ea54b4f84903b6f86ff7aee3d698, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:10.506635 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef18f44-a692-451f-9d3c-57e2739eeb0d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:10.531974 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 863/1761] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:10 2022] PUT /identity/v3/projects/ea95f83fd4114cacbc2db2d90ffc7333/users/3fa9ea54b4f84903b6f86ff7aee3d698/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:10.533740 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed0e17-4cfa-463a-90d8-0c7c27d69499 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:10.534043 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed0e17-4cfa-463a-90d8-0c7c27d69499 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:10.534310 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed0e17-4cfa-463a-90d8-0c7c27d69499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:10.556163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:10.565891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e3f57a14-b475-4675-94cf-a161d1e4fc7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3588b2dd33eb432a8390c762aa843167', 'request_id': 'req-e3f57a14-b475-4675-94cf-a161d1e4fc7b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:10.576157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e3f57a14-b475-4675-94cf-a161d1e4fc7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3588b2dd33eb432a8390c762aa843167', 'request_id': 'req-e3f57a14-b475-4675-94cf-a161d1e4fc7b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:10.588193 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e3f57a14-b475-4675-94cf-a161d1e4fc7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3588b2dd33eb432a8390c762aa843167', 'request_id': 'req-e3f57a14-b475-4675-94cf-a161d1e4fc7b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:10.608325 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-abed0e17-4cfa-463a-90d8-0c7c27d69499 None None] MFA Rules not processed for user `74b2295521bc45509ef6b68c1885ac6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:10.610926 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e3f57a14-b475-4675-94cf-a161d1e4fc7b admin admin] The token cache is being invalidate because project 3588b2dd33eb432a8390c762aa843167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:10.611454 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e3f57a14-b475-4675-94cf-a161d1e4fc7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:10.612628 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e3f57a14-b475-4675-94cf-a161d1e4fc7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:10.612899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-abed0e17-4cfa-463a-90d8-0c7c27d69499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:10.615755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 899/1762] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:10 2022] DELETE /identity/v3/projects/3588b2dd33eb432a8390c762aa843167 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:10.616809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f57a14-b475-4675-94cf-a161d1e4fc7b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:10.620821 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e3f57a14-b475-4675-94cf-a161d1e4fc7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:10.689216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 864/1763] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:10.690089 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed0e17-4cfa-463a-90d8-0c7c27d69499 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:10.698462 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-abed0e17-4cfa-463a-90d8-0c7c27d69499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:10.712736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f57a14-b475-4675-94cf-a161d1e4fc7b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:10.720738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d8984-d9c6-44b4-bd58-f2f081f31a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:10.729563 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d8984-d9c6-44b4-bd58-f2f081f31a50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:10.730064 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d8984-d9c6-44b4-bd58-f2f081f31a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:10.730532 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d8984-d9c6-44b4-bd58-f2f081f31a50 admin admin] PATH_INFO: `/v3/projects/46e7a6bb901246db8f10ec329b108d9f/users/ac1e17641ae0443396bb42a84762f4a6/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:10.733195 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d8984-d9c6-44b4-bd58-f2f081f31a50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46e7a6bb901246db8f10ec329b108d9f, user_id=ac1e17641ae0443396bb42a84762f4a6, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:10.736853 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d8984-d9c6-44b4-bd58-f2f081f31a50 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:10.752855 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 900/1764] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:10 2022] PUT /identity/v3/projects/46e7a6bb901246db8f10ec329b108d9f/users/ac1e17641ae0443396bb42a84762f4a6/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:10.754681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee845f72-afc1-4779-8d1d-edf5a2f39994 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:10.755220 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee845f72-afc1-4779-8d1d-edf5a2f39994 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:10.755873 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee845f72-afc1-4779-8d1d-edf5a2f39994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:10.783280 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed0e17-4cfa-463a-90d8-0c7c27d69499 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:10.820015 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ee845f72-afc1-4779-8d1d-edf5a2f39994 None None] MFA Rules not processed for user `3fa9ea54b4f84903b6f86ff7aee3d698`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:10.821267 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ee845f72-afc1-4779-8d1d-edf5a2f39994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:10.901245 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881b578-3457-4d66-b061-ff7630b3c696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:10.902588 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881b578-3457-4d66-b061-ff7630b3c696 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:10.902965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881b578-3457-4d66-b061-ff7630b3c696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:10.903681 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881b578-3457-4d66-b061-ff7630b3c696 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:10.904468 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881b578-3457-4d66-b061-ff7630b3c696 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:10.908124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881b578-3457-4d66-b061-ff7630b3c696 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:10.987809 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 865/1765] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:10 2022] POST /identity/v3/projects => generated 371 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:10.995311 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fd9b6-4269-4059-8a7e-2459d422e595 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:10.996752 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fd9b6-4269-4059-8a7e-2459d422e595 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:10.997048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fd9b6-4269-4059-8a7e-2459d422e595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:11.034271 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:11.038293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-c36fd9b6-4269-4059-8a7e-2459d422e595 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:11.040875 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c36fd9b6-4269-4059-8a7e-2459d422e595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:11.108056 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 901/1766] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:11.118550 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4465c-9668-40c4-b7cc-f7382ae74742 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:11.118784 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4465c-9668-40c4-b7cc-f7382ae74742 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:11.119073 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4465c-9668-40c4-b7cc-f7382ae74742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:11.233572 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 866/1767] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:11.234925 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fd9b6-4269-4059-8a7e-2459d422e595 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:11.235038 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-91c4465c-9668-40c4-b7cc-f7382ae74742 None None] MFA Rules not processed for user `ac1e17641ae0443396bb42a84762f4a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:11.241741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-91c4465c-9668-40c4-b7cc-f7382ae74742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:11.251588 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fd9b6-4269-4059-8a7e-2459d422e595 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:11.269141 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff936f79-cef7-499d-914f-fe24067081e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:11.272993 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff936f79-cef7-499d-914f-fe24067081e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:11.276006 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff936f79-cef7-499d-914f-fe24067081e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:11.276407 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff936f79-cef7-499d-914f-fe24067081e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:11.277235 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff936f79-cef7-499d-914f-fe24067081e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:11.280076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff936f79-cef7-499d-914f-fe24067081e5 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:11.354028 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 867/1768] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:11 2022] POST /identity/v3/users => generated 286 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:11.357505 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff936f79-cef7-499d-914f-fe24067081e5 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:11.365079 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ff936f79-cef7-499d-914f-fe24067081e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:11.447859 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:11.476460 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 902/1769] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:11 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:11.478061 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88dd02-6658-47c7-96bc-033111ab1297 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:11.478363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88dd02-6658-47c7-96bc-033111ab1297 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:11.478641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88dd02-6658-47c7-96bc-033111ab1297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:11.528321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-db88dd02-6658-47c7-96bc-033111ab1297 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:11.530029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-db88dd02-6658-47c7-96bc-033111ab1297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:11.564186 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 903/1770] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:11.564186 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88dd02-6658-47c7-96bc-033111ab1297 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:11.566076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff936f79-cef7-499d-914f-fe24067081e5 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:11.585420 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7e1bf-7ba4-4336-8228-d64fe66d64af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:11.586506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7e1bf-7ba4-4336-8228-d64fe66d64af service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:11.586965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7e1bf-7ba4-4336-8228-d64fe66d64af service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:11.587501 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7e1bf-7ba4-4336-8228-d64fe66d64af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:11.595898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88dd02-6658-47c7-96bc-033111ab1297 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:11.617462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2fe23-d0b3-4596-8478-c0bf66c81a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:11.618093 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2fe23-d0b3-4596-8478-c0bf66c81a3a admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:11.618551 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2fe23-d0b3-4596-8478-c0bf66c81a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:11.618731 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2fe23-d0b3-4596-8478-c0bf66c81a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:11.619473 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2fe23-d0b3-4596-8478-c0bf66c81a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:11.622051 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2fe23-d0b3-4596-8478-c0bf66c81a3a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:11.646305 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 904/1771] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:11.655435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2fe23-d0b3-4596-8478-c0bf66c81a3a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:11.683363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2fe23-d0b3-4596-8478-c0bf66c81a3a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:11.706232 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06051f-5844-4983-864f-ddb7906a650b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:11.706926 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06051f-5844-4983-864f-ddb7906a650b admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:11.707211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06051f-5844-4983-864f-ddb7906a650b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:11.707486 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06051f-5844-4983-864f-ddb7906a650b admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:11.708303 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06051f-5844-4983-864f-ddb7906a650b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:11.708614 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06051f-5844-4983-864f-ddb7906a650b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:11.710236 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06051f-5844-4983-864f-ddb7906a650b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:11.740001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7e1bf-7ba4-4336-8228-d64fe66d64af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:11.748975 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 905/1772] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:16:11.750103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06051f-5844-4983-864f-ddb7906a650b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:11.756177 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5e06051f-5844-4983-864f-ddb7906a650b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:11.758299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7e1bf-7ba4-4336-8228-d64fe66d64af service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:11.780750 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06051f-5844-4983-864f-ddb7906a650b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:11.789809 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 868/1773] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:11.790789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7e1bf-7ba4-4336-8228-d64fe66d64af service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:11.801792 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0a832-3167-4803-97ba-a643ffb0dc72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:11.806891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0a832-3167-4803-97ba-a643ffb0dc72 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:11.807121 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7e1bf-7ba4-4336-8228-d64fe66d64af service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:11.807232 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0a832-3167-4803-97ba-a643ffb0dc72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:11.807521 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0a832-3167-4803-97ba-a643ffb0dc72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:11.831436 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f239f-891d-4945-97bc-026432216af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:11.832818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f239f-891d-4945-97bc-026432216af0 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:11.833111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f239f-891d-4945-97bc-026432216af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:11.835069 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f239f-891d-4945-97bc-026432216af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:11.836697 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f239f-891d-4945-97bc-026432216af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:11.837556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0a832-3167-4803-97ba-a643ffb0dc72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:11.839030 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0a832-3167-4803-97ba-a643ffb0dc72 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:11.840711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f239f-891d-4945-97bc-026432216af0 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:11.860716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 869/1774] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:11.862685 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f239f-891d-4945-97bc-026432216af0 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:11.876894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 906/1775] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:11 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:11.887103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb659fe-b45b-45ae-ab1a-70fa53ae4e1c None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:11.887415 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb659fe-b45b-45ae-ab1a-70fa53ae4e1c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:11.887917 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f239f-891d-4945-97bc-026432216af0 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:11.891155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb659fe-b45b-45ae-ab1a-70fa53ae4e1c None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:11.893003 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 907/1776] 10.222.0.27 () {54 vars in 912 bytes} [Mon Jun 27 21:16:11 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:16:11.894140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb659fe-b45b-45ae-ab1a-70fa53ae4e1c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:11.901257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d7a83-294b-4580-9c92-d03b667b161b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:11.903640 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d7a83-294b-4580-9c92-d03b667b161b admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:11.904108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d7a83-294b-4580-9c92-d03b667b161b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:11.904423 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d7a83-294b-4580-9c92-d03b667b161b admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:11.906194 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d7a83-294b-4580-9c92-d03b667b161b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:11.910643 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d7a83-294b-4580-9c92-d03b667b161b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:11.927781 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb659fe-b45b-45ae-ab1a-70fa53ae4e1c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:11.933232 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537ca45-94f9-40c1-974f-f9a0740afec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:11.933756 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537ca45-94f9-40c1-974f-f9a0740afec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:11.934037 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537ca45-94f9-40c1-974f-f9a0740afec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:11.934357 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537ca45-94f9-40c1-974f-f9a0740afec5 admin admin] PATH_INFO: `/v3/projects/dc37ee81e325473c8d8b94d86690e098/users/99e8411bc3914e37976017a3db7403a1/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:11.953903 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537ca45-94f9-40c1-974f-f9a0740afec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc37ee81e325473c8d8b94d86690e098, user_id=99e8411bc3914e37976017a3db7403a1, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:11.958310 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537ca45-94f9-40c1-974f-f9a0740afec5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:12.016983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 870/1777] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:11 2022] POST /identity/v3/projects => generated 365 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:12.020762 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 908/1778] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:11 2022] PUT /identity/v3/projects/dc37ee81e325473c8d8b94d86690e098/users/99e8411bc3914e37976017a3db7403a1/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:12.021613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238a4fc-f0e8-48d7-95cc-fdd3b50cff45 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:12.022546 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbdb2b-d78b-4f1f-ba61-30102a6409a7 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:12.023357 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbdb2b-d78b-4f1f-ba61-30102a6409a7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:12.023475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238a4fc-f0e8-48d7-95cc-fdd3b50cff45 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:12.023690 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238a4fc-f0e8-48d7-95cc-fdd3b50cff45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:12.025913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbdb2b-d78b-4f1f-ba61-30102a6409a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:12.064986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-0238a4fc-f0e8-48d7-95cc-fdd3b50cff45 None None] MFA Rules not processed for user `e802da0b078e457da466a9a7b0f15085`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:12.066575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2cfbdb2b-d78b-4f1f-ba61-30102a6409a7 None None] MFA Rules not processed for user `9c7ce29db6b14176a4c9be436bcddc6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:12.068467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0238a4fc-f0e8-48d7-95cc-fdd3b50cff45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:12.069921 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2cfbdb2b-d78b-4f1f-ba61-30102a6409a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:12.184774 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 909/1779] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Jun 27 21:16:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:12.185749 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfbdb2b-d78b-4f1f-ba61-30102a6409a7 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:12.207210 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:12.220354 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 871/1780] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Jun 27 21:16:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:12.221768 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92689cf9-c058-43ae-837c-0ca6d74977ea None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:12.222064 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92689cf9-c058-43ae-837c-0ca6d74977ea None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:12.222345 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92689cf9-c058-43ae-837c-0ca6d74977ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:12.258943 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-92689cf9-c058-43ae-837c-0ca6d74977ea None None] MFA Rules not processed for user `99e8411bc3914e37976017a3db7403a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:12.260386 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-92689cf9-c058-43ae-837c-0ca6d74977ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:12.313056 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:12.351654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfbdb2b-d78b-4f1f-ba61-30102a6409a7 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:12.356812 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed59eb-5b8e-4efc-879f-edc3d47a6a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:12.357511 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed59eb-5b8e-4efc-879f-edc3d47a6a28 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:12.357790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed59eb-5b8e-4efc-879f-edc3d47a6a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:12.358061 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed59eb-5b8e-4efc-879f-edc3d47a6a28 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:12.358917 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed59eb-5b8e-4efc-879f-edc3d47a6a28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:12.362093 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed59eb-5b8e-4efc-879f-edc3d47a6a28 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:12.401120 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 910/1781] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:12 2022] POST /identity/v3/users => generated 283 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:12.401120 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed59eb-5b8e-4efc-879f-edc3d47a6a28 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:12.411873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 872/1782] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:12.413143 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92689cf9-c058-43ae-837c-0ca6d74977ea None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:12.415670 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-92689cf9-c058-43ae-837c-0ca6d74977ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:12.442640 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92689cf9-c058-43ae-837c-0ca6d74977ea None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:12.449503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146e388-71ec-4b2e-ba11-792d7849a4cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:12.451208 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146e388-71ec-4b2e-ba11-792d7849a4cc service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:12.451704 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146e388-71ec-4b2e-ba11-792d7849a4cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:12.452441 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146e388-71ec-4b2e-ba11-792d7849a4cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:12.466435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146e388-71ec-4b2e-ba11-792d7849a4cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:12.469580 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146e388-71ec-4b2e-ba11-792d7849a4cc service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:12.499559 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 873/1783] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:12.504953 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146e388-71ec-4b2e-ba11-792d7849a4cc service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:12.526702 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146e388-71ec-4b2e-ba11-792d7849a4cc service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:12.526924 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed59eb-5b8e-4efc-879f-edc3d47a6a28 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:12.534982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689d004-1367-4724-9cd6-7175846ceeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:12.535874 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689d004-1367-4724-9cd6-7175846ceeb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:12.536358 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a520e-7c6a-4239-867c-66ec90e5c84d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:12.536484 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689d004-1367-4724-9cd6-7175846ceeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:12.536808 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689d004-1367-4724-9cd6-7175846ceeb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:12.537750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689d004-1367-4724-9cd6-7175846ceeb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:12.538236 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a520e-7c6a-4239-867c-66ec90e5c84d service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:12.539060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a520e-7c6a-4239-867c-66ec90e5c84d service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:12.539663 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689d004-1367-4724-9cd6-7175846ceeb0 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:12.540189 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a520e-7c6a-4239-867c-66ec90e5c84d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:12.546851 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6c5a520e-7c6a-4239-867c-66ec90e5c84d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:12.556557 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 874/1784] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:12.557706 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689d004-1367-4724-9cd6-7175846ceeb0 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:12.577602 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689d004-1367-4724-9cd6-7175846ceeb0 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:12.587671 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff555f0-972f-4dc1-b92a-77fc917368b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:12.590040 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff555f0-972f-4dc1-b92a-77fc917368b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:12.590707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff555f0-972f-4dc1-b92a-77fc917368b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:12.593638 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff555f0-972f-4dc1-b92a-77fc917368b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:12.595599 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff555f0-972f-4dc1-b92a-77fc917368b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:12.600847 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff555f0-972f-4dc1-b92a-77fc917368b5 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:12.659990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 875/1785] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:12 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:12.663149 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff555f0-972f-4dc1-b92a-77fc917368b5 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:12.666757 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-eff555f0-972f-4dc1-b92a-77fc917368b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:12.708538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a520e-7c6a-4239-867c-66ec90e5c84d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:12.712025 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a520e-7c6a-4239-867c-66ec90e5c84d service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:12.736081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 911/1786] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:12 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:12.737415 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a520e-7c6a-4239-867c-66ec90e5c84d service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:12.764420 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a520e-7c6a-4239-867c-66ec90e5c84d service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:12.839050 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff555f0-972f-4dc1-b92a-77fc917368b5 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:12.852498 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9455d24-ad92-4a35-a517-5aa403e4410b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28acb0322114aebafcf62bdf9a346df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:12.853388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9455d24-ad92-4a35-a517-5aa403e4410b service swift] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:12.853705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9455d24-ad92-4a35-a517-5aa403e4410b service swift] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:12.854076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9455d24-ad92-4a35-a517-5aa403e4410b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:12.881626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:12.905550 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7b983-fe7c-42ff-ba79-cfd40c6df164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:12.906397 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7b983-fe7c-42ff-ba79-cfd40c6df164 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:12.907852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7b983-fe7c-42ff-ba79-cfd40c6df164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:12.908265 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7b983-fe7c-42ff-ba79-cfd40c6df164 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:12.910591 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7b983-fe7c-42ff-ba79-cfd40c6df164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:12.912369 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7b983-fe7c-42ff-ba79-cfd40c6df164 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:12.936629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 912/1787] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:12.936629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7b983-fe7c-42ff-ba79-cfd40c6df164 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:12.952414 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7b983-fe7c-42ff-ba79-cfd40c6df164 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:12.958814 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6a6f1-fd93-40cc-9f73-7024367402e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:12.960713 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6a6f1-fd93-40cc-9f73-7024367402e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:12.961101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6a6f1-fd93-40cc-9f73-7024367402e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:12.961873 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6a6f1-fd93-40cc-9f73-7024367402e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:12.962839 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6a6f1-fd93-40cc-9f73-7024367402e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:12.966066 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6a6f1-fd93-40cc-9f73-7024367402e0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:12.997573 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9455d24-ad92-4a35-a517-5aa403e4410b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:12.999994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9455d24-ad92-4a35-a517-5aa403e4410b service swift] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:13.009521 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 913/1788] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:12 2022] POST /identity/v3/users => generated 293 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:13.009904 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6a6f1-fd93-40cc-9f73-7024367402e0 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:13.032394 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 876/1789] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Jun 27 21:16:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:16:13.037055 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9455d24-ad92-4a35-a517-5aa403e4410b service swift] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:13.060173 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9455d24-ad92-4a35-a517-5aa403e4410b service swift] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:13.073576 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de58a60-c650-49c4-bc58-b5415225c076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:13.074619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de58a60-c650-49c4-bc58-b5415225c076 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:13.074904 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de58a60-c650-49c4-bc58-b5415225c076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:13.075233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de58a60-c650-49c4-bc58-b5415225c076 admin admin] PATH_INFO: `/v3/projects/807012405c0a4f659a06a35c0cc342cb/users/81f9256b810746e38b3d09fc2668e591/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:13.124254 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de58a60-c650-49c4-bc58-b5415225c076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=807012405c0a4f659a06a35c0cc342cb, user_id=81f9256b810746e38b3d09fc2668e591, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:13.130701 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de58a60-c650-49c4-bc58-b5415225c076 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:13.172893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:13.201045 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 877/1790] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:13 2022] PUT /identity/v3/projects/807012405c0a4f659a06a35c0cc342cb/users/81f9256b810746e38b3d09fc2668e591/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:13.214279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de58a60-c650-49c4-bc58-b5415225c076 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:13.237567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6a6f1-fd93-40cc-9f73-7024367402e0 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:13.394961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de58a60-c650-49c4-bc58-b5415225c076 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:13.403910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47ee3c-49d3-490d-bd19-e2de9a2acf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:13.404675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47ee3c-49d3-490d-bd19-e2de9a2acf56 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:13.406497 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47ee3c-49d3-490d-bd19-e2de9a2acf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:13.406902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47ee3c-49d3-490d-bd19-e2de9a2acf56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:13.407959 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c47ee3c-49d3-490d-bd19-e2de9a2acf56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:13.413799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c47ee3c-49d3-490d-bd19-e2de9a2acf56 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:13.452042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 878/1791] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:13.454549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89131edd-b0ae-4511-b0fd-daa85ef425c1 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:13.455062 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89131edd-b0ae-4511-b0fd-daa85ef425c1 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:13.455544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89131edd-b0ae-4511-b0fd-daa85ef425c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:13.531033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-89131edd-b0ae-4511-b0fd-daa85ef425c1 None None] MFA Rules not processed for user `81f9256b810746e38b3d09fc2668e591`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:13.534437 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-89131edd-b0ae-4511-b0fd-daa85ef425c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:13.636566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a40ca-3c8c-46f1-9d35-d5be45c4c989 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:13.638934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a40ca-3c8c-46f1-9d35-d5be45c4c989 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:13.639240 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a40ca-3c8c-46f1-9d35-d5be45c4c989 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:13.639522 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a40ca-3c8c-46f1-9d35-d5be45c4c989 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:13.649164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-332a40ca-3c8c-46f1-9d35-d5be45c4c989 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:13.745603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 879/1792] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:13 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:13.749677 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89131edd-b0ae-4511-b0fd-daa85ef425c1 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:13.774857 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89131edd-b0ae-4511-b0fd-daa85ef425c1 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:13.792423 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3074c34-c432-4032-9935-32f4ccaf19e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:13.794251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3074c34-c432-4032-9935-32f4ccaf19e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:13.794567 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3074c34-c432-4032-9935-32f4ccaf19e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:13.795075 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3074c34-c432-4032-9935-32f4ccaf19e3 admin admin] PATH_INFO: `/v3/projects/60a0162eac2b41edb17e7cefbe96526f/users/e072b0de4cc64a03acaa4ab8742b5de4/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:13.832056 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3074c34-c432-4032-9935-32f4ccaf19e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60a0162eac2b41edb17e7cefbe96526f, user_id=e072b0de4cc64a03acaa4ab8742b5de4, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:13.835255 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3074c34-c432-4032-9935-32f4ccaf19e3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:13.858118 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332a40ca-3c8c-46f1-9d35-d5be45c4c989 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:13.862528 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332a40ca-3c8c-46f1-9d35-d5be45c4c989 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:13.866484 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 880/1793] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:13 2022] PUT /identity/v3/projects/60a0162eac2b41edb17e7cefbe96526f/users/e072b0de4cc64a03acaa4ab8742b5de4/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:13.867621 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3074c34-c432-4032-9935-32f4ccaf19e3 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:13.902289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 914/1794] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 894 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:13.903468 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a40ca-3c8c-46f1-9d35-d5be45c4c989 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:13.939213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a40ca-3c8c-46f1-9d35-d5be45c4c989 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:14.036802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:14.048845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3074c34-c432-4032-9935-32f4ccaf19e3 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:14.062806 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db077437-21e5-48f6-90bc-e52ba01873ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:14.063500 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db077437-21e5-48f6-90bc-e52ba01873ac admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:14.063850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db077437-21e5-48f6-90bc-e52ba01873ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:14.064082 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db077437-21e5-48f6-90bc-e52ba01873ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:14.065178 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db077437-21e5-48f6-90bc-e52ba01873ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:14.069235 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9001123-595f-4482-ac70-ca11d1231c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:14.070514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9001123-595f-4482-ac70-ca11d1231c85 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:14.070818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db077437-21e5-48f6-90bc-e52ba01873ac admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:14.071461 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9001123-595f-4482-ac70-ca11d1231c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:14.072126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9001123-595f-4482-ac70-ca11d1231c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:14.073223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9001123-595f-4482-ac70-ca11d1231c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:14.075928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9001123-595f-4482-ac70-ca11d1231c85 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:14.098172 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:14.129459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 915/1795] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:14.132029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9001123-595f-4482-ac70-ca11d1231c85 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:14.159077 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 881/1796] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:13 2022] POST /identity/v3/projects => generated 361 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:14.164844 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db077437-21e5-48f6-90bc-e52ba01873ac admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:14.185998 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9001123-595f-4482-ac70-ca11d1231c85 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:14.322234 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db077437-21e5-48f6-90bc-e52ba01873ac admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:14.336574 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157fc73-cb9d-4a35-a126-2b328a1ea58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:14.337279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157fc73-cb9d-4a35-a126-2b328a1ea58c admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:14.337553 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157fc73-cb9d-4a35-a126-2b328a1ea58c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:14.337831 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157fc73-cb9d-4a35-a126-2b328a1ea58c admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:14.338537 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4157fc73-cb9d-4a35-a126-2b328a1ea58c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:14.340523 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4157fc73-cb9d-4a35-a126-2b328a1ea58c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:14.350862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee4895-a69f-4a38-8a5d-f93434bf7153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:14.353630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee4895-a69f-4a38-8a5d-f93434bf7153 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:14.355296 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee4895-a69f-4a38-8a5d-f93434bf7153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:14.355877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee4895-a69f-4a38-8a5d-f93434bf7153 admin admin] PATH_INFO: `/v3/projects/60a0162eac2b41edb17e7cefbe96526f/users/e072b0de4cc64a03acaa4ab8742b5de4/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:14.362620 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee4895-a69f-4a38-8a5d-f93434bf7153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60a0162eac2b41edb17e7cefbe96526f, user_id=e072b0de4cc64a03acaa4ab8742b5de4, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:14.371153 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee4895-a69f-4a38-8a5d-f93434bf7153 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:14.378383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 882/1797] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:14 2022] POST /identity/v3/users => generated 287 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:14.382764 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157fc73-cb9d-4a35-a126-2b328a1ea58c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:14.414573 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157fc73-cb9d-4a35-a126-2b328a1ea58c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:14.451469 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 916/1798] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:14 2022] PUT /identity/v3/projects/60a0162eac2b41edb17e7cefbe96526f/users/e072b0de4cc64a03acaa4ab8742b5de4/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:14.466607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c7193-b05b-45ad-a039-b487df637843 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:14.466902 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c7193-b05b-45ad-a039-b487df637843 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:14.467185 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c7193-b05b-45ad-a039-b487df637843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:14.544374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def450b-ac31-4207-8a46-23f4e3a56336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:14.545077 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def450b-ac31-4207-8a46-23f4e3a56336 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:14.545360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def450b-ac31-4207-8a46-23f4e3a56336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:14.545636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def450b-ac31-4207-8a46-23f4e3a56336 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:14.546494 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def450b-ac31-4207-8a46-23f4e3a56336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:14.562245 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def450b-ac31-4207-8a46-23f4e3a56336 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:14.596243 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-886c7193-b05b-45ad-a039-b487df637843 None None] MFA Rules not processed for user `e072b0de4cc64a03acaa4ab8742b5de4`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:14.609343 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-886c7193-b05b-45ad-a039-b487df637843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:14.620645 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 883/1799] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:14.621694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def450b-ac31-4207-8a46-23f4e3a56336 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:14.665893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def450b-ac31-4207-8a46-23f4e3a56336 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:14.710768 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d082f-6fa1-4f95-a722-84a387d8326a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:14.711883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d082f-6fa1-4f95-a722-84a387d8326a admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:14.712387 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d082f-6fa1-4f95-a722-84a387d8326a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:14.714591 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d082f-6fa1-4f95-a722-84a387d8326a admin admin] PATH_INFO: `/v3/projects/f4a9c77596a94deda399342c53b22092/users/41c4f6977f30483089610842c2865fdf/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:14.828939 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d082f-6fa1-4f95-a722-84a387d8326a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4a9c77596a94deda399342c53b22092, user_id=41c4f6977f30483089610842c2865fdf, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:14.846304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d082f-6fa1-4f95-a722-84a387d8326a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:14.915128 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 884/1800] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:14 2022] PUT /identity/v3/projects/f4a9c77596a94deda399342c53b22092/users/41c4f6977f30483089610842c2865fdf/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:14.919656 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42394219-c768-434e-a283-df5717b7a1a1 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:14.921772 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42394219-c768-434e-a283-df5717b7a1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:14.924014 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42394219-c768-434e-a283-df5717b7a1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:15.025243 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-42394219-c768-434e-a283-df5717b7a1a1 None None] MFA Rules not processed for user `9c7ce29db6b14176a4c9be436bcddc6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:15.030471 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-42394219-c768-434e-a283-df5717b7a1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:15.038649 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 917/1801] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 573 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:15.043133 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c7193-b05b-45ad-a039-b487df637843 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:15.102054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c7193-b05b-45ad-a039-b487df637843 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:15.165388 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:15.208102 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:15.213702 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a932608-fc14-48b0-9e3d-5ff576b6d16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:15.216618 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a932608-fc14-48b0-9e3d-5ff576b6d16b admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:15.216899 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a932608-fc14-48b0-9e3d-5ff576b6d16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:15.217181 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a932608-fc14-48b0-9e3d-5ff576b6d16b admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:15.217932 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a932608-fc14-48b0-9e3d-5ff576b6d16b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:15.236740 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a932608-fc14-48b0-9e3d-5ff576b6d16b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:15.249252 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 885/1802] 10.222.0.46 () {58 vars in 1015 bytes} [Mon Jun 27 21:16:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:15.251101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42394219-c768-434e-a283-df5717b7a1a1 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:15.271068 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 918/1803] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:15.272913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c11b3-c84c-465e-9180-a218d75a9b65 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:15.273407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c11b3-c84c-465e-9180-a218d75a9b65 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:15.276004 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c11b3-c84c-465e-9180-a218d75a9b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:15.331578 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-dd1c11b3-c84c-465e-9180-a218d75a9b65 None None] MFA Rules not processed for user `30cbde26a1964f22902c24917006fb90`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:15.335419 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-dd1c11b3-c84c-465e-9180-a218d75a9b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:15.411274 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42394219-c768-434e-a283-df5717b7a1a1 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:15.416084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9537b0a-5ff8-4c28-b234-fe954b067a68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:15.416895 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9537b0a-5ff8-4c28-b234-fe954b067a68 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:15.417172 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9537b0a-5ff8-4c28-b234-fe954b067a68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:15.417478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9537b0a-5ff8-4c28-b234-fe954b067a68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:15.495687 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 919/1804] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:15.496789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c11b3-c84c-465e-9180-a218d75a9b65 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:15.552695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c11b3-c84c-465e-9180-a218d75a9b65 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:15.567244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584eeb83-e89f-4cf9-b95b-e713638d1ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:15.568298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584eeb83-e89f-4cf9-b95b-e713638d1ee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:15.568659 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584eeb83-e89f-4cf9-b95b-e713638d1ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:15.568963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584eeb83-e89f-4cf9-b95b-e713638d1ee8 admin admin] PATH_INFO: `/v3/projects/f4a9c77596a94deda399342c53b22092/users/41c4f6977f30483089610842c2865fdf/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:15.571036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584eeb83-e89f-4cf9-b95b-e713638d1ee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4a9c77596a94deda399342c53b22092, user_id=41c4f6977f30483089610842c2865fdf, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:15.574976 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584eeb83-e89f-4cf9-b95b-e713638d1ee8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:15.604736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 920/1805] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:15 2022] PUT /identity/v3/projects/f4a9c77596a94deda399342c53b22092/users/41c4f6977f30483089610842c2865fdf/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:15.606523 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584eeb83-e89f-4cf9-b95b-e713638d1ee8 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:15.608950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-584eeb83-e89f-4cf9-b95b-e713638d1ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:15.708940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9537b0a-5ff8-4c28-b234-fe954b067a68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:15.710597 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9537b0a-5ff8-4c28-b234-fe954b067a68 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:15.748769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 886/1806] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:15.750154 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f794a-1aea-4d50-90e2-c95e4a846494 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:15.750464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f794a-1aea-4d50-90e2-c95e4a846494 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:15.750738 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f794a-1aea-4d50-90e2-c95e4a846494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:15.792158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d94f794a-1aea-4d50-90e2-c95e4a846494 None None] MFA Rules not processed for user `41c4f6977f30483089610842c2865fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:15.793778 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d94f794a-1aea-4d50-90e2-c95e4a846494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:15.883124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584eeb83-e89f-4cf9-b95b-e713638d1ee8 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:15.907314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3438e05-9b0f-4a9b-ad42-9c6fe5fa709c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:15.910950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3438e05-9b0f-4a9b-ad42-9c6fe5fa709c service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:15.911244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3438e05-9b0f-4a9b-ad42-9c6fe5fa709c service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:15.913532 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3438e05-9b0f-4a9b-ad42-9c6fe5fa709c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:15.950874 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3438e05-9b0f-4a9b-ad42-9c6fe5fa709c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:15.956156 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3438e05-9b0f-4a9b-ad42-9c6fe5fa709c service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:16.033957 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 921/1807] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:16.035938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3438e05-9b0f-4a9b-ad42-9c6fe5fa709c service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:16.128350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 887/1808] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:15 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:16.204588 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3438e05-9b0f-4a9b-ad42-9c6fe5fa709c service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:16.215858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984bbf4-3bc2-45ee-aa72-379d84ed4e37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:16.216869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984bbf4-3bc2-45ee-aa72-379d84ed4e37 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:16.217164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984bbf4-3bc2-45ee-aa72-379d84ed4e37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:16.218076 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984bbf4-3bc2-45ee-aa72-379d84ed4e37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:16.317445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:16.338333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984bbf4-3bc2-45ee-aa72-379d84ed4e37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:16.342539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984bbf4-3bc2-45ee-aa72-379d84ed4e37 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:16.385289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 922/1809] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:16.569532 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f794a-1aea-4d50-90e2-c95e4a846494 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:16.586940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f794a-1aea-4d50-90e2-c95e4a846494 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:16.592858 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba29c3-a5ea-4bfa-8ccb-a9df5d1986fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:16.593707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba29c3-a5ea-4bfa-8ccb-a9df5d1986fd service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:16.593991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba29c3-a5ea-4bfa-8ccb-a9df5d1986fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:16.594254 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba29c3-a5ea-4bfa-8ccb-a9df5d1986fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:16.600155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-93ba29c3-a5ea-4bfa-8ccb-a9df5d1986fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:16.678688 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:16.757489 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba29c3-a5ea-4bfa-8ccb-a9df5d1986fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:16.761728 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba29c3-a5ea-4bfa-8ccb-a9df5d1986fd service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:16.803782 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 888/1810] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:16 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:16.843285 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984bbf4-3bc2-45ee-aa72-379d84ed4e37 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:16.861840 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984bbf4-3bc2-45ee-aa72-379d84ed4e37 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:16.866939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82434c79-fb03-4466-9372-70b112dc3fd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:16.867595 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82434c79-fb03-4466-9372-70b112dc3fd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:16.868057 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82434c79-fb03-4466-9372-70b112dc3fd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:16.868614 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82434c79-fb03-4466-9372-70b112dc3fd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:16.883742 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82434c79-fb03-4466-9372-70b112dc3fd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:16.885622 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82434c79-fb03-4466-9372-70b112dc3fd4 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:16.925314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 923/1811] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:16 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:17.241739 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba29c3-a5ea-4bfa-8ccb-a9df5d1986fd service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:17.252028 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba29c3-a5ea-4bfa-8ccb-a9df5d1986fd service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:17.257453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5872d3-a2f3-4c34-bea4-8c63b45d2251 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:17.258294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5872d3-a2f3-4c34-bea4-8c63b45d2251 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:17.258787 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5872d3-a2f3-4c34-bea4-8c63b45d2251 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:17.259246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5872d3-a2f3-4c34-bea4-8c63b45d2251 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:17.345838 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5872d3-a2f3-4c34-bea4-8c63b45d2251 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:17.347300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5872d3-a2f3-4c34-bea4-8c63b45d2251 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:17.358450 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 889/1812] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:17 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:17.794979 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b60a9-8504-495e-b6e0-b1a04e10b269 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:17.795571 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b60a9-8504-495e-b6e0-b1a04e10b269 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:17.795571 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b60a9-8504-495e-b6e0-b1a04e10b269 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:17.797275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 924/1813] 10.222.0.27 () {54 vars in 921 bytes} [Mon Jun 27 21:16:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:16:17.808716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6386a-effc-44b8-a826-5ae5ae6737dd None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:17.809228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6386a-effc-44b8-a826-5ae5ae6737dd None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:17.809319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6386a-effc-44b8-a826-5ae5ae6737dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:17.830102 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-edf6386a-effc-44b8-a826-5ae5ae6737dd None None] MFA Rules not processed for user `8d7fe01ef64f46f6a0e77d5945219b33`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:17.832240 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-edf6386a-effc-44b8-a826-5ae5ae6737dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:17.903843 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 890/1814] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Jun 27 21:16:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:17.915696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b60a9-8504-495e-b6e0-b1a04e10b269 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:17.917334 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fb0b60a9-8504-495e-b6e0-b1a04e10b269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:17.934969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b60a9-8504-495e-b6e0-b1a04e10b269 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:17.941049 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0afcdd-2535-44de-9a01-8e0bcd8c0b0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:17.941713 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0afcdd-2535-44de-9a01-8e0bcd8c0b0e service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:17.941997 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0afcdd-2535-44de-9a01-8e0bcd8c0b0e service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:17.942276 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0afcdd-2535-44de-9a01-8e0bcd8c0b0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:17.951823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0afcdd-2535-44de-9a01-8e0bcd8c0b0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:17.953507 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0afcdd-2535-44de-9a01-8e0bcd8c0b0e service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:17.964241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 925/1815] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:25.580452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6386a-effc-44b8-a826-5ae5ae6737dd None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:25.670480 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:25.679445 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6386a-effc-44b8-a826-5ae5ae6737dd None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:25.686454 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e097dfd-7e45-4a2b-b10b-066b92c476ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:25.687580 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e097dfd-7e45-4a2b-b10b-066b92c476ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:25.688063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e097dfd-7e45-4a2b-b10b-066b92c476ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:25.688341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e097dfd-7e45-4a2b-b10b-066b92c476ed admin admin] PATH_INFO: `/v3/users/9d4b5bdccab6438ab42203c259ec6860` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:25.690029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e097dfd-7e45-4a2b-b10b-066b92c476ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d4b5bdccab6438ab42203c259ec6860)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:25.692409 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e097dfd-7e45-4a2b-b10b-066b92c476ed admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:25.749087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9e097dfd-7e45-4a2b-b10b-066b92c476ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d4b5bdccab6438ab42203c259ec6860'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:25.749736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9e097dfd-7e45-4a2b-b10b-066b92c476ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d4b5bdccab6438ab42203c259ec6860'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:25.750399 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9e097dfd-7e45-4a2b-b10b-066b92c476ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d4b5bdccab6438ab42203c259ec6860'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:25.763961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9e097dfd-7e45-4a2b-b10b-066b92c476ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d4b5bdccab6438ab42203c259ec6860'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:25.771408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9e097dfd-7e45-4a2b-b10b-066b92c476ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d4b5bdccab6438ab42203c259ec6860'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:25.788282 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 891/1816] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:25 2022] DELETE /identity/v3/users/9d4b5bdccab6438ab42203c259ec6860 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:25.793627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5b8bb-6496-451c-9e58-ad016f353a4c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:25.794373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5b8bb-6496-451c-9e58-ad016f353a4c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:25.794373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5b8bb-6496-451c-9e58-ad016f353a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:25.813463 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-fee5b8bb-6496-451c-9e58-ad016f353a4c None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:25.814308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fee5b8bb-6496-451c-9e58-ad016f353a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:25.873432 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 926/1817] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:25.884241 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e097dfd-7e45-4a2b-b10b-066b92c476ed admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:25.885644 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9e097dfd-7e45-4a2b-b10b-066b92c476ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:25.950706 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e097dfd-7e45-4a2b-b10b-066b92c476ed admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:25.956364 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89addc0-dea8-4a32-bd3b-e25f291e939d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:25.957019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89addc0-dea8-4a32-bd3b-e25f291e939d service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:25.957303 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89addc0-dea8-4a32-bd3b-e25f291e939d service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:25.957592 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89addc0-dea8-4a32-bd3b-e25f291e939d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:25.969599 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89addc0-dea8-4a32-bd3b-e25f291e939d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:25.971063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89addc0-dea8-4a32-bd3b-e25f291e939d service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:25.981375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 892/1818] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:26.283855 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5b8bb-6496-451c-9e58-ad016f353a4c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:26.285385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fee5b8bb-6496-451c-9e58-ad016f353a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:26.298891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5b8bb-6496-451c-9e58-ad016f353a4c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:26.304025 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60f11a-84a3-45ae-a59c-d643ff85dbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:26.304916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60f11a-84a3-45ae-a59c-d643ff85dbe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:26.305369 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60f11a-84a3-45ae-a59c-d643ff85dbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:26.305479 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60f11a-84a3-45ae-a59c-d643ff85dbe6 admin admin] PATH_INFO: `/v3/projects/77e08e720d64447fabb7a05eba60d92a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:26.306777 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60f11a-84a3-45ae-a59c-d643ff85dbe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77e08e720d64447fabb7a05eba60d92a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:26.308766 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60f11a-84a3-45ae-a59c-d643ff85dbe6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:26.359954 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ba60f11a-84a3-45ae-a59c-d643ff85dbe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77e08e720d64447fabb7a05eba60d92a', 'request_id': 'req-ba60f11a-84a3-45ae-a59c-d643ff85dbe6'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:26.370545 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ba60f11a-84a3-45ae-a59c-d643ff85dbe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77e08e720d64447fabb7a05eba60d92a', 'request_id': 'req-ba60f11a-84a3-45ae-a59c-d643ff85dbe6'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:26.377880 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ba60f11a-84a3-45ae-a59c-d643ff85dbe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77e08e720d64447fabb7a05eba60d92a', 'request_id': 'req-ba60f11a-84a3-45ae-a59c-d643ff85dbe6'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:26.390248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ba60f11a-84a3-45ae-a59c-d643ff85dbe6 admin admin] The token cache is being invalidate because project 77e08e720d64447fabb7a05eba60d92a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:26.390657 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ba60f11a-84a3-45ae-a59c-d643ff85dbe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:26.391451 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ba60f11a-84a3-45ae-a59c-d643ff85dbe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:26.393162 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 927/1819] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:26 2022] DELETE /identity/v3/projects/77e08e720d64447fabb7a05eba60d92a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:26.398305 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89addc0-dea8-4a32-bd3b-e25f291e939d service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:26.408853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89addc0-dea8-4a32-bd3b-e25f291e939d service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:26.469045 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62c6ec-edc4-43cf-aa59-d38522a32083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:26.469927 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62c6ec-edc4-43cf-aa59-d38522a32083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:26.470208 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62c6ec-edc4-43cf-aa59-d38522a32083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:26.470483 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62c6ec-edc4-43cf-aa59-d38522a32083 admin admin] PATH_INFO: `/v3/users/ac1e17641ae0443396bb42a84762f4a6` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:26.471778 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62c6ec-edc4-43cf-aa59-d38522a32083 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac1e17641ae0443396bb42a84762f4a6)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:26.473697 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62c6ec-edc4-43cf-aa59-d38522a32083 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:26.494147 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:26.523221 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4a62c6ec-edc4-43cf-aa59-d38522a32083 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac1e17641ae0443396bb42a84762f4a6'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:26.523911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4a62c6ec-edc4-43cf-aa59-d38522a32083 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac1e17641ae0443396bb42a84762f4a6'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:26.524523 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4a62c6ec-edc4-43cf-aa59-d38522a32083 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac1e17641ae0443396bb42a84762f4a6'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:26.537356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4a62c6ec-edc4-43cf-aa59-d38522a32083 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac1e17641ae0443396bb42a84762f4a6'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:26.544749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4a62c6ec-edc4-43cf-aa59-d38522a32083 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac1e17641ae0443396bb42a84762f4a6'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:26.558558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 893/1820] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:26 2022] DELETE /identity/v3/users/ac1e17641ae0443396bb42a84762f4a6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:26.964741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60f11a-84a3-45ae-a59c-d643ff85dbe6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:26.966210 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ba60f11a-84a3-45ae-a59c-d643ff85dbe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:26.979226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60f11a-84a3-45ae-a59c-d643ff85dbe6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:27.026177 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:27.045155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65b366-2ed1-4d35-b702-2f09b224dc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:27.046099 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65b366-2ed1-4d35-b702-2f09b224dc1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:27.046380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65b366-2ed1-4d35-b702-2f09b224dc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:27.046657 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65b366-2ed1-4d35-b702-2f09b224dc1c admin admin] PATH_INFO: `/v3/projects/46e7a6bb901246db8f10ec329b108d9f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:27.047854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65b366-2ed1-4d35-b702-2f09b224dc1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=46e7a6bb901246db8f10ec329b108d9f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:27.049793 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65b366-2ed1-4d35-b702-2f09b224dc1c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:27.098084 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5e65b366-2ed1-4d35-b702-2f09b224dc1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46e7a6bb901246db8f10ec329b108d9f', 'request_id': 'req-5e65b366-2ed1-4d35-b702-2f09b224dc1c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:27.105864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5e65b366-2ed1-4d35-b702-2f09b224dc1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46e7a6bb901246db8f10ec329b108d9f', 'request_id': 'req-5e65b366-2ed1-4d35-b702-2f09b224dc1c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:27.111246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5e65b366-2ed1-4d35-b702-2f09b224dc1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46e7a6bb901246db8f10ec329b108d9f', 'request_id': 'req-5e65b366-2ed1-4d35-b702-2f09b224dc1c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:27.122208 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5e65b366-2ed1-4d35-b702-2f09b224dc1c admin admin] The token cache is being invalidate because project 46e7a6bb901246db8f10ec329b108d9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:27.122492 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5e65b366-2ed1-4d35-b702-2f09b224dc1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:27.123117 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5e65b366-2ed1-4d35-b702-2f09b224dc1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:27.124760 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 928/1821] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:26 2022] DELETE /identity/v3/projects/46e7a6bb901246db8f10ec329b108d9f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:27.129623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b244c74-26d9-464e-8ca5-f8267130612d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:27.130077 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b244c74-26d9-464e-8ca5-f8267130612d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:27.130215 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b244c74-26d9-464e-8ca5-f8267130612d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:27.151430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-9b244c74-26d9-464e-8ca5-f8267130612d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:27.152693 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9b244c74-26d9-464e-8ca5-f8267130612d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:27.344177 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 894/1822] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:27.351862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc75bd-bf5a-4840-b2a9-90c1e2e509af None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:27.352151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc75bd-bf5a-4840-b2a9-90c1e2e509af None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:27.352424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc75bd-bf5a-4840-b2a9-90c1e2e509af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:27.370743 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-36bc75bd-bf5a-4840-b2a9-90c1e2e509af None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:27.372006 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-36bc75bd-bf5a-4840-b2a9-90c1e2e509af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:27.380867 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 929/1823] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:27.385755 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b244c74-26d9-464e-8ca5-f8267130612d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:27.398037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b244c74-26d9-464e-8ca5-f8267130612d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:27.404093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713df20-d44f-4ad8-8fe2-00d92391b132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:27.404740 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713df20-d44f-4ad8-8fe2-00d92391b132 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:27.405021 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713df20-d44f-4ad8-8fe2-00d92391b132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:27.405497 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713df20-d44f-4ad8-8fe2-00d92391b132 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:27.406226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713df20-d44f-4ad8-8fe2-00d92391b132 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:27.406532 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713df20-d44f-4ad8-8fe2-00d92391b132 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:27.408173 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713df20-d44f-4ad8-8fe2-00d92391b132 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:27.424029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 895/1824] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:16:27.425503 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc75bd-bf5a-4840-b2a9-90c1e2e509af None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:27.434493 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc75bd-bf5a-4840-b2a9-90c1e2e509af None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:27.439787 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e35d6c-2718-4110-b32c-475d5fb224af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:27.440457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e35d6c-2718-4110-b32c-475d5fb224af admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:27.440734 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e35d6c-2718-4110-b32c-475d5fb224af admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:27.441003 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e35d6c-2718-4110-b32c-475d5fb224af admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:27.441683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e35d6c-2718-4110-b32c-475d5fb224af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:27.443584 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e35d6c-2718-4110-b32c-475d5fb224af admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:27.469597 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 930/1825] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:27 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:27.474093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713df20-d44f-4ad8-8fe2-00d92391b132 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:27.484744 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713df20-d44f-4ad8-8fe2-00d92391b132 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:27.541300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3f7de-f765-4235-b9b2-6e6b1cb575e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:27.541976 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3f7de-f765-4235-b9b2-6e6b1cb575e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:27.542256 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3f7de-f765-4235-b9b2-6e6b1cb575e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:27.542525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3f7de-f765-4235-b9b2-6e6b1cb575e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:27.543229 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3f7de-f765-4235-b9b2-6e6b1cb575e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:27.545210 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3f7de-f765-4235-b9b2-6e6b1cb575e2 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:27.570089 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 896/1826] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:27 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:27.571843 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e35d6c-2718-4110-b32c-475d5fb224af admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:27.583183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e35d6c-2718-4110-b32c-475d5fb224af admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:27.587843 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf23a25-461f-44ce-a5e9-f809fa3aebc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:27.588508 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf23a25-461f-44ce-a5e9-f809fa3aebc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:27.588790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf23a25-461f-44ce-a5e9-f809fa3aebc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:27.589060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf23a25-461f-44ce-a5e9-f809fa3aebc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:27.589778 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf23a25-461f-44ce-a5e9-f809fa3aebc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:27.591404 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf23a25-461f-44ce-a5e9-f809fa3aebc5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:27.601906 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 931/1827] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:27.605764 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3f7de-f765-4235-b9b2-6e6b1cb575e2 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:27.616657 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3f7de-f765-4235-b9b2-6e6b1cb575e2 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:27.623145 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217b696-ebeb-47a6-b675-47adc2977dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:27.624041 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217b696-ebeb-47a6-b675-47adc2977dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:27.624351 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217b696-ebeb-47a6-b675-47adc2977dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:27.624628 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217b696-ebeb-47a6-b675-47adc2977dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:27.625454 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217b696-ebeb-47a6-b675-47adc2977dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:27.627261 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217b696-ebeb-47a6-b675-47adc2977dd9 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:27.638606 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 897/1828] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:27.642802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf23a25-461f-44ce-a5e9-f809fa3aebc5 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:27.653184 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf23a25-461f-44ce-a5e9-f809fa3aebc5 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:27.657999 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008f2c4-4667-4762-ac16-07a1b47dd133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:27.658889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008f2c4-4667-4762-ac16-07a1b47dd133 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:27.659170 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008f2c4-4667-4762-ac16-07a1b47dd133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:27.659445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008f2c4-4667-4762-ac16-07a1b47dd133 admin admin] PATH_INFO: `/v3/projects/828460b6b1e74475a73198375ede31cc/users/ab45045e52444612b6a62138283ba955/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:27.674249 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008f2c4-4667-4762-ac16-07a1b47dd133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=828460b6b1e74475a73198375ede31cc, user_id=ab45045e52444612b6a62138283ba955, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:27.676940 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008f2c4-4667-4762-ac16-07a1b47dd133 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:27.691592 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 932/1829] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:27 2022] PUT /identity/v3/projects/828460b6b1e74475a73198375ede31cc/users/ab45045e52444612b6a62138283ba955/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:27.703294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c9048-576e-4cdc-8cdb-d52af240aa33 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:27.703586 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c9048-576e-4cdc-8cdb-d52af240aa33 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:27.703913 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c9048-576e-4cdc-8cdb-d52af240aa33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:27.710327 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:27.734388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-186c9048-576e-4cdc-8cdb-d52af240aa33 None None] MFA Rules not processed for user `ab45045e52444612b6a62138283ba955`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:27.735627 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-186c9048-576e-4cdc-8cdb-d52af240aa33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:27.813663 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 898/1830] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:27.818374 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008f2c4-4667-4762-ac16-07a1b47dd133 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:27.880373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008f2c4-4667-4762-ac16-07a1b47dd133 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:27.885144 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64595a8-d8f6-43f8-9240-800cef890cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:27.885810 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64595a8-d8f6-43f8-9240-800cef890cbe admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:27.886097 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64595a8-d8f6-43f8-9240-800cef890cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:27.886376 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64595a8-d8f6-43f8-9240-800cef890cbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:27.887081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64595a8-d8f6-43f8-9240-800cef890cbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:27.889041 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64595a8-d8f6-43f8-9240-800cef890cbe admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:27.900121 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:27.918820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 933/1831] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:27 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:27.923673 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c9048-576e-4cdc-8cdb-d52af240aa33 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:27.935260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c9048-576e-4cdc-8cdb-d52af240aa33 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:27.998649 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbd8f5-9474-49f1-aee1-2aff3e0cf140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:27.999444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbd8f5-9474-49f1-aee1-2aff3e0cf140 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:27.999767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbd8f5-9474-49f1-aee1-2aff3e0cf140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:28.000043 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbd8f5-9474-49f1-aee1-2aff3e0cf140 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:28.000736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dbd8f5-9474-49f1-aee1-2aff3e0cf140 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:28.002644 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dbd8f5-9474-49f1-aee1-2aff3e0cf140 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:28.028721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 899/1832] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:27 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:28.030392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64595a8-d8f6-43f8-9240-800cef890cbe admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:28.040967 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64595a8-d8f6-43f8-9240-800cef890cbe admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:28.045908 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fae08f-103d-4a20-bdef-8fae6343dbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:28.046645 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fae08f-103d-4a20-bdef-8fae6343dbe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:28.046994 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fae08f-103d-4a20-bdef-8fae6343dbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:28.047337 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fae08f-103d-4a20-bdef-8fae6343dbe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:28.048168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fae08f-103d-4a20-bdef-8fae6343dbe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:28.049888 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fae08f-103d-4a20-bdef-8fae6343dbe9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:28.061751 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 934/1833] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:28.066178 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbd8f5-9474-49f1-aee1-2aff3e0cf140 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:28.076299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbd8f5-9474-49f1-aee1-2aff3e0cf140 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:28.081466 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4e254-d8c4-4ac1-a55f-0cf9f8ae6267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:28.082410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4e254-d8c4-4ac1-a55f-0cf9f8ae6267 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:28.082691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4e254-d8c4-4ac1-a55f-0cf9f8ae6267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:28.082966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4e254-d8c4-4ac1-a55f-0cf9f8ae6267 admin admin] PATH_INFO: `/v3/projects/303853dbfc2a4f1a8165c9abd3a031f2/users/6c2caa953829470eb5aaffa885e9970c/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:28.105232 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4e254-d8c4-4ac1-a55f-0cf9f8ae6267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=303853dbfc2a4f1a8165c9abd3a031f2, user_id=6c2caa953829470eb5aaffa885e9970c, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:28.107967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4e254-d8c4-4ac1-a55f-0cf9f8ae6267 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:28.121831 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 900/1834] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:28 2022] PUT /identity/v3/projects/303853dbfc2a4f1a8165c9abd3a031f2/users/6c2caa953829470eb5aaffa885e9970c/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:28.126083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fae08f-103d-4a20-bdef-8fae6343dbe9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:28.136291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fae08f-103d-4a20-bdef-8fae6343dbe9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:28.193989 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9457f7-e822-4a7d-a591-7ab085b0f436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:28.194652 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9457f7-e822-4a7d-a591-7ab085b0f436 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:28.194928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9457f7-e822-4a7d-a591-7ab085b0f436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:28.195200 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9457f7-e822-4a7d-a591-7ab085b0f436 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:28.195986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9457f7-e822-4a7d-a591-7ab085b0f436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:28.197597 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9457f7-e822-4a7d-a591-7ab085b0f436 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:28.207805 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 935/1835] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:16:28.212302 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4e254-d8c4-4ac1-a55f-0cf9f8ae6267 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:28.223593 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4e254-d8c4-4ac1-a55f-0cf9f8ae6267 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:28.228897 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e284f-1ccf-4f1f-9b33-b787ab80ae45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:28.229850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e284f-1ccf-4f1f-9b33-b787ab80ae45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:28.230132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e284f-1ccf-4f1f-9b33-b787ab80ae45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:28.230410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e284f-1ccf-4f1f-9b33-b787ab80ae45 admin admin] PATH_INFO: `/v3/projects/303853dbfc2a4f1a8165c9abd3a031f2/users/6c2caa953829470eb5aaffa885e9970c/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:28.232450 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e284f-1ccf-4f1f-9b33-b787ab80ae45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=303853dbfc2a4f1a8165c9abd3a031f2, user_id=6c2caa953829470eb5aaffa885e9970c, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:28.235100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e284f-1ccf-4f1f-9b33-b787ab80ae45 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:28.246659 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 901/1836] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:28 2022] PUT /identity/v3/projects/303853dbfc2a4f1a8165c9abd3a031f2/users/6c2caa953829470eb5aaffa885e9970c/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:28.256987 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ae895-1f92-4f1c-9ad6-64c3e1029141 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:28.257325 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ae895-1f92-4f1c-9ad6-64c3e1029141 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:28.257605 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ae895-1f92-4f1c-9ad6-64c3e1029141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:28.284723 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5e5ae895-1f92-4f1c-9ad6-64c3e1029141 None None] MFA Rules not processed for user `6c2caa953829470eb5aaffa885e9970c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:28.286217 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5e5ae895-1f92-4f1c-9ad6-64c3e1029141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:28.355666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 936/1837] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:28 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:28.367160 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e284f-1ccf-4f1f-9b33-b787ab80ae45 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:28.369054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8a8e284f-1ccf-4f1f-9b33-b787ab80ae45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:28.418768 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:28.432601 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e284f-1ccf-4f1f-9b33-b787ab80ae45 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:28.443778 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d08030-f831-485a-b23d-0786b342589b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:28.444488 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d08030-f831-485a-b23d-0786b342589b service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:28.444774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d08030-f831-485a-b23d-0786b342589b service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:28.445054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d08030-f831-485a-b23d-0786b342589b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:28.455078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d08030-f831-485a-b23d-0786b342589b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:28.456556 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d08030-f831-485a-b23d-0786b342589b service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:28.465957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 902/1838] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:28 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:28.550476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ae895-1f92-4f1c-9ad6-64c3e1029141 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:28.560327 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ae895-1f92-4f1c-9ad6-64c3e1029141 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:28.564811 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b34b4-79f9-4b0f-848f-99f7bc967cc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:28.565672 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b34b4-79f9-4b0f-848f-99f7bc967cc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:28.566043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b34b4-79f9-4b0f-848f-99f7bc967cc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:28.566408 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b34b4-79f9-4b0f-848f-99f7bc967cc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:28.625426 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b34b4-79f9-4b0f-848f-99f7bc967cc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:28.628770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b34b4-79f9-4b0f-848f-99f7bc967cc2 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:28.636089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:28.641285 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 937/1839] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:28 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:35.887713 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d08030-f831-485a-b23d-0786b342589b service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:35.889368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-61d08030-f831-485a-b23d-0786b342589b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:35.900812 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d08030-f831-485a-b23d-0786b342589b service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:35.904444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483d558-a793-4d4d-978e-58654af7b627 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:35.905179 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483d558-a793-4d4d-978e-58654af7b627 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:35.905398 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483d558-a793-4d4d-978e-58654af7b627 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:35.905729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483d558-a793-4d4d-978e-58654af7b627 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:35.907476 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e483d558-a793-4d4d-978e-58654af7b627 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:35.983403 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483d558-a793-4d4d-978e-58654af7b627 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:35.986106 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483d558-a793-4d4d-978e-58654af7b627 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:36.003056 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 903/1840] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:38.624435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a787ad4-30f9-4e4c-b441-21fba319a697 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:38.624971 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a787ad4-30f9-4e4c-b441-21fba319a697 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:38.624971 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a787ad4-30f9-4e4c-b441-21fba319a697 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:38.626486 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 938/1841] 10.222.0.27 () {54 vars in 921 bytes} [Mon Jun 27 21:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:16:38.634833 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89a396-1976-4855-8ffc-a0148255b929 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:38.635792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89a396-1976-4855-8ffc-a0148255b929 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:38.636210 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89a396-1976-4855-8ffc-a0148255b929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:38.669084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-dd89a396-1976-4855-8ffc-a0148255b929 None None] MFA Rules not processed for user `8d7fe01ef64f46f6a0e77d5945219b33`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:38.670705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-dd89a396-1976-4855-8ffc-a0148255b929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:38.822765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 904/1842] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Jun 27 21:16:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:38.837559 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a787ad4-30f9-4e4c-b441-21fba319a697 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:38.839774 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7a787ad4-30f9-4e4c-b441-21fba319a697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:38.874981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a787ad4-30f9-4e4c-b441-21fba319a697 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:38.887552 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f263dd2-9a76-48f9-822f-90da50516588 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:38.888763 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f263dd2-9a76-48f9-822f-90da50516588 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:38.889592 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f263dd2-9a76-48f9-822f-90da50516588 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:38.890037 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f263dd2-9a76-48f9-822f-90da50516588 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:38.904992 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f263dd2-9a76-48f9-822f-90da50516588 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:38.909120 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f263dd2-9a76-48f9-822f-90da50516588 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:38.926757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 939/1843] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:39.374368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89a396-1976-4855-8ffc-a0148255b929 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:39.375929 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-dd89a396-1976-4855-8ffc-a0148255b929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:39.445256 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89a396-1976-4855-8ffc-a0148255b929 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:39.451400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d403c-4fdb-4364-9591-fb0a41c78c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:39.452345 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d403c-4fdb-4364-9591-fb0a41c78c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:39.452594 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d403c-4fdb-4364-9591-fb0a41c78c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:39.452865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d403c-4fdb-4364-9591-fb0a41c78c54 admin admin] PATH_INFO: `/v3/users/f64baf6ff7be4ea38ed5b7b6fdd05cc7` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:39.454202 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d403c-4fdb-4364-9591-fb0a41c78c54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f64baf6ff7be4ea38ed5b7b6fdd05cc7)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:39.456554 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d403c-4fdb-4364-9591-fb0a41c78c54 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:39.482976 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:39.510652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-cb1d403c-4fdb-4364-9591-fb0a41c78c54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f64baf6ff7be4ea38ed5b7b6fdd05cc7'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:39.511322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-cb1d403c-4fdb-4364-9591-fb0a41c78c54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f64baf6ff7be4ea38ed5b7b6fdd05cc7'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:39.511934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-cb1d403c-4fdb-4364-9591-fb0a41c78c54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f64baf6ff7be4ea38ed5b7b6fdd05cc7'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:39.525128 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-cb1d403c-4fdb-4364-9591-fb0a41c78c54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f64baf6ff7be4ea38ed5b7b6fdd05cc7'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:39.532633 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-cb1d403c-4fdb-4364-9591-fb0a41c78c54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f64baf6ff7be4ea38ed5b7b6fdd05cc7'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:39.551236 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 905/1844] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:39 2022] DELETE /identity/v3/users/f64baf6ff7be4ea38ed5b7b6fdd05cc7 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:39.555526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee36add-8038-46a8-9496-24662a65d3ce None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:39.555943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee36add-8038-46a8-9496-24662a65d3ce None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:39.556293 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee36add-8038-46a8-9496-24662a65d3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:39.574211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-cee36add-8038-46a8-9496-24662a65d3ce None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:39.575451 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cee36add-8038-46a8-9496-24662a65d3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:39.636639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 940/1845] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:39.646716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d403c-4fdb-4364-9591-fb0a41c78c54 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:39.648472 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-cb1d403c-4fdb-4364-9591-fb0a41c78c54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:39.706054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d403c-4fdb-4364-9591-fb0a41c78c54 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:39.709662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c993a9e-2e22-4886-8776-434aecc269f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:39.710468 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c993a9e-2e22-4886-8776-434aecc269f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:39.710746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c993a9e-2e22-4886-8776-434aecc269f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:39.711140 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c993a9e-2e22-4886-8776-434aecc269f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:39.720909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c993a9e-2e22-4886-8776-434aecc269f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:39.722381 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c993a9e-2e22-4886-8776-434aecc269f9 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:39.732325 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 906/1846] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:40.017704 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee36add-8038-46a8-9496-24662a65d3ce None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:40.019760 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cee36add-8038-46a8-9496-24662a65d3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:40.039635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee36add-8038-46a8-9496-24662a65d3ce None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:40.045497 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c8fa8-aa62-4340-b2a8-283fb42f3ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:40.046476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c8fa8-aa62-4340-b2a8-283fb42f3ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:40.046758 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c8fa8-aa62-4340-b2a8-283fb42f3ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:40.047032 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c8fa8-aa62-4340-b2a8-283fb42f3ac2 admin admin] PATH_INFO: `/v3/projects/fbd72b1384ec4fccaef7b8dbb2418a0d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:40.048487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c8fa8-aa62-4340-b2a8-283fb42f3ac2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbd72b1384ec4fccaef7b8dbb2418a0d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:40.050629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c8fa8-aa62-4340-b2a8-283fb42f3ac2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:40.102042 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4d8c8fa8-aa62-4340-b2a8-283fb42f3ac2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbd72b1384ec4fccaef7b8dbb2418a0d', 'request_id': 'req-4d8c8fa8-aa62-4340-b2a8-283fb42f3ac2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.112361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4d8c8fa8-aa62-4340-b2a8-283fb42f3ac2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbd72b1384ec4fccaef7b8dbb2418a0d', 'request_id': 'req-4d8c8fa8-aa62-4340-b2a8-283fb42f3ac2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.119790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4d8c8fa8-aa62-4340-b2a8-283fb42f3ac2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbd72b1384ec4fccaef7b8dbb2418a0d', 'request_id': 'req-4d8c8fa8-aa62-4340-b2a8-283fb42f3ac2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.133481 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4d8c8fa8-aa62-4340-b2a8-283fb42f3ac2 admin admin] The token cache is being invalidate because project fbd72b1384ec4fccaef7b8dbb2418a0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:40.133841 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4d8c8fa8-aa62-4340-b2a8-283fb42f3ac2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.134560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4d8c8fa8-aa62-4340-b2a8-283fb42f3ac2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.139284 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 941/1847] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:40 2022] DELETE /identity/v3/projects/fbd72b1384ec4fccaef7b8dbb2418a0d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:40.140822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c993a9e-2e22-4886-8776-434aecc269f9 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:40.150694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c993a9e-2e22-4886-8776-434aecc269f9 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:40.209387 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61eb4c-3b89-4463-8fed-3113e843a84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:40.210280 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61eb4c-3b89-4463-8fed-3113e843a84e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:40.210567 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61eb4c-3b89-4463-8fed-3113e843a84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:40.210846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61eb4c-3b89-4463-8fed-3113e843a84e admin admin] PATH_INFO: `/v3/users/16ff0377c9e14dda8cd7b5e65cc23986` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:40.212061 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61eb4c-3b89-4463-8fed-3113e843a84e admin admin] RBAC: Authorizing `identity:delete_user(user_id=16ff0377c9e14dda8cd7b5e65cc23986)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:40.214007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61eb4c-3b89-4463-8fed-3113e843a84e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:40.256553 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:40.265477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3b61eb4c-3b89-4463-8fed-3113e843a84e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16ff0377c9e14dda8cd7b5e65cc23986'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.266488 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3b61eb4c-3b89-4463-8fed-3113e843a84e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16ff0377c9e14dda8cd7b5e65cc23986'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.267126 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3b61eb4c-3b89-4463-8fed-3113e843a84e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16ff0377c9e14dda8cd7b5e65cc23986'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.283283 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3b61eb4c-3b89-4463-8fed-3113e843a84e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16ff0377c9e14dda8cd7b5e65cc23986'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.291394 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3b61eb4c-3b89-4463-8fed-3113e843a84e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16ff0377c9e14dda8cd7b5e65cc23986'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.309450 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 907/1848] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:40 2022] DELETE /identity/v3/users/16ff0377c9e14dda8cd7b5e65cc23986 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:40.641703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c8fa8-aa62-4340-b2a8-283fb42f3ac2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:40.643588 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4d8c8fa8-aa62-4340-b2a8-283fb42f3ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:40.665225 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c8fa8-aa62-4340-b2a8-283fb42f3ac2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:40.722207 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b3ae2-c53f-406a-b78e-6d8872d0145f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:40.723082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b3ae2-c53f-406a-b78e-6d8872d0145f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:40.723356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b3ae2-c53f-406a-b78e-6d8872d0145f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:40.723639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b3ae2-c53f-406a-b78e-6d8872d0145f admin admin] PATH_INFO: `/v3/projects/b99f36b12f8642dbaa1b951102994fcf` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:40.724958 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b3ae2-c53f-406a-b78e-6d8872d0145f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b99f36b12f8642dbaa1b951102994fcf)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:40.726846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b3ae2-c53f-406a-b78e-6d8872d0145f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:40.738184 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:40.782061 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e65b3ae2-c53f-406a-b78e-6d8872d0145f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b99f36b12f8642dbaa1b951102994fcf', 'request_id': 'req-e65b3ae2-c53f-406a-b78e-6d8872d0145f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.792694 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e65b3ae2-c53f-406a-b78e-6d8872d0145f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b99f36b12f8642dbaa1b951102994fcf', 'request_id': 'req-e65b3ae2-c53f-406a-b78e-6d8872d0145f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.800567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e65b3ae2-c53f-406a-b78e-6d8872d0145f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b99f36b12f8642dbaa1b951102994fcf', 'request_id': 'req-e65b3ae2-c53f-406a-b78e-6d8872d0145f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.812341 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e65b3ae2-c53f-406a-b78e-6d8872d0145f admin admin] The token cache is being invalidate because project b99f36b12f8642dbaa1b951102994fcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:40.812622 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e65b3ae2-c53f-406a-b78e-6d8872d0145f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.813317 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e65b3ae2-c53f-406a-b78e-6d8872d0145f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.814933 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 942/1849] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:40 2022] DELETE /identity/v3/projects/b99f36b12f8642dbaa1b951102994fcf => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:40.819332 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61eb4c-3b89-4463-8fed-3113e843a84e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:40.820842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3b61eb4c-3b89-4463-8fed-3113e843a84e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:40.834371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61eb4c-3b89-4463-8fed-3113e843a84e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:40.893408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06eb60b-a132-4c76-a245-67708935cb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:40.894300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06eb60b-a132-4c76-a245-67708935cb58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:40.894584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06eb60b-a132-4c76-a245-67708935cb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:40.894861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06eb60b-a132-4c76-a245-67708935cb58 admin admin] PATH_INFO: `/v3/users/3fa9ea54b4f84903b6f86ff7aee3d698` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:40.896124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06eb60b-a132-4c76-a245-67708935cb58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fa9ea54b4f84903b6f86ff7aee3d698)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:40.898073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06eb60b-a132-4c76-a245-67708935cb58 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:40.947386 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c06eb60b-a132-4c76-a245-67708935cb58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fa9ea54b4f84903b6f86ff7aee3d698'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.948069 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c06eb60b-a132-4c76-a245-67708935cb58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fa9ea54b4f84903b6f86ff7aee3d698'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.948717 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c06eb60b-a132-4c76-a245-67708935cb58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fa9ea54b4f84903b6f86ff7aee3d698'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.960552 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c06eb60b-a132-4c76-a245-67708935cb58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fa9ea54b4f84903b6f86ff7aee3d698'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.967817 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c06eb60b-a132-4c76-a245-67708935cb58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fa9ea54b4f84903b6f86ff7aee3d698'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.983633 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 908/1850] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:40 2022] DELETE /identity/v3/users/3fa9ea54b4f84903b6f86ff7aee3d698 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:41.281373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b3ae2-c53f-406a-b78e-6d8872d0145f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:41.282888 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e65b3ae2-c53f-406a-b78e-6d8872d0145f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:41.312070 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b3ae2-c53f-406a-b78e-6d8872d0145f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:41.369120 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3c987-33f1-43bc-a5de-96e6fdb2aee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:41.370039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3c987-33f1-43bc-a5de-96e6fdb2aee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:41.370321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3c987-33f1-43bc-a5de-96e6fdb2aee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:41.370594 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3c987-33f1-43bc-a5de-96e6fdb2aee4 admin admin] PATH_INFO: `/v3/projects/ea95f83fd4114cacbc2db2d90ffc7333` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:41.371889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3c987-33f1-43bc-a5de-96e6fdb2aee4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea95f83fd4114cacbc2db2d90ffc7333)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:41.373890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3c987-33f1-43bc-a5de-96e6fdb2aee4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:41.429997 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c0e3c987-33f1-43bc-a5de-96e6fdb2aee4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea95f83fd4114cacbc2db2d90ffc7333', 'request_id': 'req-c0e3c987-33f1-43bc-a5de-96e6fdb2aee4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:41.440316 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c0e3c987-33f1-43bc-a5de-96e6fdb2aee4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea95f83fd4114cacbc2db2d90ffc7333', 'request_id': 'req-c0e3c987-33f1-43bc-a5de-96e6fdb2aee4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:41.450072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c0e3c987-33f1-43bc-a5de-96e6fdb2aee4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea95f83fd4114cacbc2db2d90ffc7333', 'request_id': 'req-c0e3c987-33f1-43bc-a5de-96e6fdb2aee4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:41.482894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c0e3c987-33f1-43bc-a5de-96e6fdb2aee4 admin admin] The token cache is being invalidate because project ea95f83fd4114cacbc2db2d90ffc7333 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:41.483202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c0e3c987-33f1-43bc-a5de-96e6fdb2aee4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:41.485268 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c0e3c987-33f1-43bc-a5de-96e6fdb2aee4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:41.493869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 943/1851] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:41 2022] DELETE /identity/v3/projects/ea95f83fd4114cacbc2db2d90ffc7333 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:41.498463 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3ea34-f756-4a17-b578-e2d05083d65a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:41.498879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3ea34-f756-4a17-b578-e2d05083d65a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:41.499193 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3ea34-f756-4a17-b578-e2d05083d65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:41.521053 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ebe3ea34-f756-4a17-b578-e2d05083d65a None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:41.522307 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ebe3ea34-f756-4a17-b578-e2d05083d65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:41.565121 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:41.624716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 909/1852] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:41.626880 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbaecc9-8dd1-4e9a-8352-3bcb09d0a5da None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:41.627322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbaecc9-8dd1-4e9a-8352-3bcb09d0a5da None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:41.627799 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbaecc9-8dd1-4e9a-8352-3bcb09d0a5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:41.649910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4cbaecc9-8dd1-4e9a-8352-3bcb09d0a5da None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:41.651828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4cbaecc9-8dd1-4e9a-8352-3bcb09d0a5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:41.661783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 944/1853] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:41.666292 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3ea34-f756-4a17-b578-e2d05083d65a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:41.677733 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3ea34-f756-4a17-b578-e2d05083d65a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:41.682722 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc887c5b-cd7a-4d74-ab64-201cc3d422df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:41.683594 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc887c5b-cd7a-4d74-ab64-201cc3d422df admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:41.684039 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc887c5b-cd7a-4d74-ab64-201cc3d422df admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:41.684483 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc887c5b-cd7a-4d74-ab64-201cc3d422df admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:41.685358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc887c5b-cd7a-4d74-ab64-201cc3d422df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:41.685860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc887c5b-cd7a-4d74-ab64-201cc3d422df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:41.687781 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc887c5b-cd7a-4d74-ab64-201cc3d422df admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:41.704944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 910/1854] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:16:41.706107 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbaecc9-8dd1-4e9a-8352-3bcb09d0a5da None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:41.715719 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbaecc9-8dd1-4e9a-8352-3bcb09d0a5da None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:41.721960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7c5b63-a6c2-47c5-91fb-963bc60bb470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:41.722997 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c5b63-a6c2-47c5-91fb-963bc60bb470 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:41.723415 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c5b63-a6c2-47c5-91fb-963bc60bb470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:41.724009 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c5b63-a6c2-47c5-91fb-963bc60bb470 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:41.724962 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7c5b63-a6c2-47c5-91fb-963bc60bb470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:41.727314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7c5b63-a6c2-47c5-91fb-963bc60bb470 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:41.756864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 945/1855] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:41 2022] POST /identity/v3/projects => generated 361 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:41.760950 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc887c5b-cd7a-4d74-ab64-201cc3d422df admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:41.772416 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc887c5b-cd7a-4d74-ab64-201cc3d422df admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:41.840456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a80e4a-a045-40d7-a39c-1ebb01acf433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:41.840456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a80e4a-a045-40d7-a39c-1ebb01acf433 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:41.840456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a80e4a-a045-40d7-a39c-1ebb01acf433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:41.840456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a80e4a-a045-40d7-a39c-1ebb01acf433 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:41.841292 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a80e4a-a045-40d7-a39c-1ebb01acf433 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:41.841292 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a80e4a-a045-40d7-a39c-1ebb01acf433 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:41.861860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 911/1856] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:41 2022] POST /identity/v3/users => generated 281 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:41.865919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7c5b63-a6c2-47c5-91fb-963bc60bb470 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:41.871889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:41.878502 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7c5b63-a6c2-47c5-91fb-963bc60bb470 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:41.886735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec67069-1d15-4209-b1dd-d2fc5dff2bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:41.887563 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec67069-1d15-4209-b1dd-d2fc5dff2bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:41.887982 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec67069-1d15-4209-b1dd-d2fc5dff2bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:41.888343 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec67069-1d15-4209-b1dd-d2fc5dff2bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:41.889360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec67069-1d15-4209-b1dd-d2fc5dff2bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:41.891499 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec67069-1d15-4209-b1dd-d2fc5dff2bb0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:41.903047 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 946/1857] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:41.907959 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a80e4a-a045-40d7-a39c-1ebb01acf433 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:41.918488 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a80e4a-a045-40d7-a39c-1ebb01acf433 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:41.925707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6e40b-435f-4519-b531-1b1017aa678a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:41.926401 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6e40b-435f-4519-b531-1b1017aa678a admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:41.926687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6e40b-435f-4519-b531-1b1017aa678a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:41.927078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6e40b-435f-4519-b531-1b1017aa678a admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:41.927844 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6e40b-435f-4519-b531-1b1017aa678a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:41.929488 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6e40b-435f-4519-b531-1b1017aa678a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:41.943045 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 912/1858] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:41.947712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec67069-1d15-4209-b1dd-d2fc5dff2bb0 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:41.958817 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec67069-1d15-4209-b1dd-d2fc5dff2bb0 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:41.963639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14113f7-f9a7-49c9-823f-884335a4a2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:41.964551 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14113f7-f9a7-49c9-823f-884335a4a2cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:41.964966 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14113f7-f9a7-49c9-823f-884335a4a2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:41.965243 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14113f7-f9a7-49c9-823f-884335a4a2cf admin admin] PATH_INFO: `/v3/projects/9dda080b2b6f42d6a66e0dd0d0916538/users/e7378c12a1304378aee782d098e3bf00/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:41.981489 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14113f7-f9a7-49c9-823f-884335a4a2cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dda080b2b6f42d6a66e0dd0d0916538, user_id=e7378c12a1304378aee782d098e3bf00, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:41.984379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14113f7-f9a7-49c9-823f-884335a4a2cf admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:42.003439 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 947/1859] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:41 2022] PUT /identity/v3/projects/9dda080b2b6f42d6a66e0dd0d0916538/users/e7378c12a1304378aee782d098e3bf00/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:42.015259 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e3a64-fbc8-4aec-968e-852547ff106d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:42.015549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e3a64-fbc8-4aec-968e-852547ff106d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:42.015876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e3a64-fbc8-4aec-968e-852547ff106d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:42.048848 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-cb1e3a64-fbc8-4aec-968e-852547ff106d None None] MFA Rules not processed for user `e7378c12a1304378aee782d098e3bf00`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:42.050227 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-cb1e3a64-fbc8-4aec-968e-852547ff106d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:42.120077 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 913/1860] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:42.152202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14113f7-f9a7-49c9-823f-884335a4a2cf admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:42.154110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b14113f7-f9a7-49c9-823f-884335a4a2cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:42.212379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14113f7-f9a7-49c9-823f-884335a4a2cf admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:42.216773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37157a90-8917-4748-8b00-31f749330742 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:42.217612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37157a90-8917-4748-8b00-31f749330742 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:42.217954 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37157a90-8917-4748-8b00-31f749330742 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:42.218261 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37157a90-8917-4748-8b00-31f749330742 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:42.227483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37157a90-8917-4748-8b00-31f749330742 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:42.229029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37157a90-8917-4748-8b00-31f749330742 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:42.238336 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 948/1861] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:42 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:42.252464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e3a64-fbc8-4aec-968e-852547ff106d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:42.296827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:42.332224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e3a64-fbc8-4aec-968e-852547ff106d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:42.336784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb41111-5c30-472a-9ddb-4fb9d275ea35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:42.337672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb41111-5c30-472a-9ddb-4fb9d275ea35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:42.337951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb41111-5c30-472a-9ddb-4fb9d275ea35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:42.338224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb41111-5c30-472a-9ddb-4fb9d275ea35 admin admin] PATH_INFO: `/v3/users/e7378c12a1304378aee782d098e3bf00` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:42.339428 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb41111-5c30-472a-9ddb-4fb9d275ea35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7378c12a1304378aee782d098e3bf00)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:42.341375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb41111-5c30-472a-9ddb-4fb9d275ea35 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:42.391562 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8bb41111-5c30-472a-9ddb-4fb9d275ea35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7378c12a1304378aee782d098e3bf00'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:42.392229 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8bb41111-5c30-472a-9ddb-4fb9d275ea35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7378c12a1304378aee782d098e3bf00'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:42.393222 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8bb41111-5c30-472a-9ddb-4fb9d275ea35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7378c12a1304378aee782d098e3bf00'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:42.405355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8bb41111-5c30-472a-9ddb-4fb9d275ea35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7378c12a1304378aee782d098e3bf00'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:42.412916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8bb41111-5c30-472a-9ddb-4fb9d275ea35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7378c12a1304378aee782d098e3bf00'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:42.429013 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 914/1862] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:42 2022] DELETE /identity/v3/users/e7378c12a1304378aee782d098e3bf00 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:42.433729 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4672ed-8b5a-40cf-bcc0-2d330adf4790 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:42.434032 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4672ed-8b5a-40cf-bcc0-2d330adf4790 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:42.434307 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4672ed-8b5a-40cf-bcc0-2d330adf4790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:42.453861 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6c4672ed-8b5a-40cf-bcc0-2d330adf4790 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:42.455153 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6c4672ed-8b5a-40cf-bcc0-2d330adf4790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:42.517567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 949/1863] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:42.527593 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb41111-5c30-472a-9ddb-4fb9d275ea35 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:42.529052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8bb41111-5c30-472a-9ddb-4fb9d275ea35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:42.608085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb41111-5c30-472a-9ddb-4fb9d275ea35 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:42.612571 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01ecd8-1e43-416b-bd50-c06f6e5c060c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:42.613326 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01ecd8-1e43-416b-bd50-c06f6e5c060c service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:42.613745 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01ecd8-1e43-416b-bd50-c06f6e5c060c service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:42.614057 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01ecd8-1e43-416b-bd50-c06f6e5c060c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:42.623433 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01ecd8-1e43-416b-bd50-c06f6e5c060c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:42.624881 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01ecd8-1e43-416b-bd50-c06f6e5c060c service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:42.634296 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 915/1864] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:43.077426 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4672ed-8b5a-40cf-bcc0-2d330adf4790 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:43.078932 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6c4672ed-8b5a-40cf-bcc0-2d330adf4790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:43.094219 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4672ed-8b5a-40cf-bcc0-2d330adf4790 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:43.104960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f100f-597f-4c8d-83bf-71d2666f3dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:43.105932 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f100f-597f-4c8d-83bf-71d2666f3dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:43.106343 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f100f-597f-4c8d-83bf-71d2666f3dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:43.106812 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f100f-597f-4c8d-83bf-71d2666f3dae admin admin] PATH_INFO: `/v3/projects/9dda080b2b6f42d6a66e0dd0d0916538` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:43.108224 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f100f-597f-4c8d-83bf-71d2666f3dae admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dda080b2b6f42d6a66e0dd0d0916538)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:43.110295 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f100f-597f-4c8d-83bf-71d2666f3dae admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:43.161044 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-652f100f-597f-4c8d-83bf-71d2666f3dae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dda080b2b6f42d6a66e0dd0d0916538', 'request_id': 'req-652f100f-597f-4c8d-83bf-71d2666f3dae'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:43.169812 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-652f100f-597f-4c8d-83bf-71d2666f3dae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dda080b2b6f42d6a66e0dd0d0916538', 'request_id': 'req-652f100f-597f-4c8d-83bf-71d2666f3dae'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:43.176454 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-652f100f-597f-4c8d-83bf-71d2666f3dae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dda080b2b6f42d6a66e0dd0d0916538', 'request_id': 'req-652f100f-597f-4c8d-83bf-71d2666f3dae'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:43.177511 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:43.189368 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-652f100f-597f-4c8d-83bf-71d2666f3dae admin admin] The token cache is being invalidate because project 9dda080b2b6f42d6a66e0dd0d0916538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:43.189704 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-652f100f-597f-4c8d-83bf-71d2666f3dae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:43.191048 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-652f100f-597f-4c8d-83bf-71d2666f3dae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:43.192893 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 950/1865] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:43 2022] DELETE /identity/v3/projects/9dda080b2b6f42d6a66e0dd0d0916538 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:43.198728 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e866cec-6c1b-4c62-857e-4ddd5783e7d0 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:43.199211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e866cec-6c1b-4c62-857e-4ddd5783e7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:43.199298 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e866cec-6c1b-4c62-857e-4ddd5783e7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:43.218013 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6e866cec-6c1b-4c62-857e-4ddd5783e7d0 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:43.219231 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6e866cec-6c1b-4c62-857e-4ddd5783e7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:43.272910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:43.281690 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f100f-597f-4c8d-83bf-71d2666f3dae admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:43.283259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-652f100f-597f-4c8d-83bf-71d2666f3dae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:43.291751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 916/1866] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:43.294455 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1da4ff-3671-4f94-acb3-df76561e082d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:43.294747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1da4ff-3671-4f94-acb3-df76561e082d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:43.295060 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1da4ff-3671-4f94-acb3-df76561e082d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:43.296799 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f100f-597f-4c8d-83bf-71d2666f3dae admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:43.301563 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fded4c7-ac98-4ed7-bf6e-98a84bc847e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:43.302515 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fded4c7-ac98-4ed7-bf6e-98a84bc847e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:43.302864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fded4c7-ac98-4ed7-bf6e-98a84bc847e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:43.303202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fded4c7-ac98-4ed7-bf6e-98a84bc847e1 admin admin] PATH_INFO: `/v3/users/ab45045e52444612b6a62138283ba955` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:43.304572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fded4c7-ac98-4ed7-bf6e-98a84bc847e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab45045e52444612b6a62138283ba955)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:43.306576 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fded4c7-ac98-4ed7-bf6e-98a84bc847e1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:43.313483 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6b1da4ff-3671-4f94-acb3-df76561e082d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:43.314704 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6b1da4ff-3671-4f94-acb3-df76561e082d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:43.322663 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 917/1867] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:43.327798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1da4ff-3671-4f94-acb3-df76561e082d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:43.337698 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1da4ff-3671-4f94-acb3-df76561e082d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:43.343135 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947515a-fcab-4efb-97ac-9556fe7d1d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:43.344251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947515a-fcab-4efb-97ac-9556fe7d1d86 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:43.344877 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947515a-fcab-4efb-97ac-9556fe7d1d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:43.345041 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947515a-fcab-4efb-97ac-9556fe7d1d86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:43.345932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8947515a-fcab-4efb-97ac-9556fe7d1d86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:43.346432 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8947515a-fcab-4efb-97ac-9556fe7d1d86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:43.348003 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8947515a-fcab-4efb-97ac-9556fe7d1d86 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:43.361738 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 918/1868] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:16:43.366417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947515a-fcab-4efb-97ac-9556fe7d1d86 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:43.372746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0fded4c7-ac98-4ed7-bf6e-98a84bc847e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab45045e52444612b6a62138283ba955'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:43.373605 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0fded4c7-ac98-4ed7-bf6e-98a84bc847e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab45045e52444612b6a62138283ba955'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:43.374347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0fded4c7-ac98-4ed7-bf6e-98a84bc847e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab45045e52444612b6a62138283ba955'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:43.379508 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947515a-fcab-4efb-97ac-9556fe7d1d86 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:43.386204 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402aed6-e170-4eea-836a-6a9c8663b38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:43.386880 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402aed6-e170-4eea-836a-6a9c8663b38a admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:43.387240 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402aed6-e170-4eea-836a-6a9c8663b38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:43.387521 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402aed6-e170-4eea-836a-6a9c8663b38a admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:43.388313 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402aed6-e170-4eea-836a-6a9c8663b38a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:43.388657 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0fded4c7-ac98-4ed7-bf6e-98a84bc847e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab45045e52444612b6a62138283ba955'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:43.390800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402aed6-e170-4eea-836a-6a9c8663b38a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:43.397559 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0fded4c7-ac98-4ed7-bf6e-98a84bc847e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab45045e52444612b6a62138283ba955'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:43.415193 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 951/1869] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:43 2022] DELETE /identity/v3/users/ab45045e52444612b6a62138283ba955 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:43.419511 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ed107-c258-4f07-9b8d-265f0cd4252c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:43.419969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ed107-c258-4f07-9b8d-265f0cd4252c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:43.420335 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ed107-c258-4f07-9b8d-265f0cd4252c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:43.421376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 919/1870] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:43 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:43.425535 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402aed6-e170-4eea-836a-6a9c8663b38a admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:43.427427 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6402aed6-e170-4eea-836a-6a9c8663b38a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:43.442555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-487ed107-c258-4f07-9b8d-265f0cd4252c None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:43.443801 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-487ed107-c258-4f07-9b8d-265f0cd4252c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:43.519303 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402aed6-e170-4eea-836a-6a9c8663b38a admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:43.524774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81aa612-1c50-485d-98ff-74134a69db30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:43.526130 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aa612-1c50-485d-98ff-74134a69db30 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:43.526528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aa612-1c50-485d-98ff-74134a69db30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:43.526758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aa612-1c50-485d-98ff-74134a69db30 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:43.527452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81aa612-1c50-485d-98ff-74134a69db30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:43.529577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81aa612-1c50-485d-98ff-74134a69db30 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:43.535961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 952/1871] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:43.544468 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ed107-c258-4f07-9b8d-265f0cd4252c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:43.546643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-487ed107-c258-4f07-9b8d-265f0cd4252c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:43.557885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 920/1872] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:43 2022] POST /identity/v3/users => generated 286 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:43.562049 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81aa612-1c50-485d-98ff-74134a69db30 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:43.596042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81aa612-1c50-485d-98ff-74134a69db30 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:43.611102 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fe3c2-71e1-4e87-8697-fe6f0a050239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:43.612151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fe3c2-71e1-4e87-8697-fe6f0a050239 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:43.614477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fe3c2-71e1-4e87-8697-fe6f0a050239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:43.614855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fe3c2-71e1-4e87-8697-fe6f0a050239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:43.615771 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fe3c2-71e1-4e87-8697-fe6f0a050239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:43.618969 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fe3c2-71e1-4e87-8697-fe6f0a050239 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:43.645906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 921/1873] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:43.647401 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fe3c2-71e1-4e87-8697-fe6f0a050239 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:43.655597 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ed107-c258-4f07-9b8d-265f0cd4252c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:43.661161 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591e43d-badd-48ce-b832-1b29c6834cbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:43.662909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591e43d-badd-48ce-b832-1b29c6834cbb service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:43.663300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591e43d-badd-48ce-b832-1b29c6834cbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:43.664490 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591e43d-badd-48ce-b832-1b29c6834cbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:43.674673 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fe3c2-71e1-4e87-8697-fe6f0a050239 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:43.687062 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffc4c0-a198-4220-b37d-5f28ef086c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:43.690604 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffc4c0-a198-4220-b37d-5f28ef086c0c admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:43.691073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffc4c0-a198-4220-b37d-5f28ef086c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:43.691477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffc4c0-a198-4220-b37d-5f28ef086c0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:43.693308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ffc4c0-a198-4220-b37d-5f28ef086c0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:43.700667 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ffc4c0-a198-4220-b37d-5f28ef086c0c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:43.705284 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591e43d-badd-48ce-b832-1b29c6834cbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:43.713096 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591e43d-badd-48ce-b832-1b29c6834cbb service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:43.747447 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 922/1874] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:16:43.748918 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54034776-0987-4fc5-90ea-3f829b1f9d49 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:43.749349 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54034776-0987-4fc5-90ea-3f829b1f9d49 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:43.749796 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 953/1875] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:43.750176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54034776-0987-4fc5-90ea-3f829b1f9d49 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:43.751197 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591e43d-badd-48ce-b832-1b29c6834cbb service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:43.754512 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 923/1876] 10.222.0.27 () {54 vars in 921 bytes} [Mon Jun 27 21:16:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:16:43.770316 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591e43d-badd-48ce-b832-1b29c6834cbb service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:43.784675 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f91b8-d844-4a7e-9c80-c1f3fdb28f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:43.785891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f91b8-d844-4a7e-9c80-c1f3fdb28f0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:43.786439 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f91b8-d844-4a7e-9c80-c1f3fdb28f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:43.786835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f91b8-d844-4a7e-9c80-c1f3fdb28f0e admin admin] PATH_INFO: `/v3/projects/54cbfe298d34475cbf86b5c0be99acda/users/2fbdcaa712b84abdba95a8eb6c8843da/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:43.794908 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:43.810905 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f91b8-d844-4a7e-9c80-c1f3fdb28f0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=54cbfe298d34475cbf86b5c0be99acda, user_id=2fbdcaa712b84abdba95a8eb6c8843da, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:43.815408 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f91b8-d844-4a7e-9c80-c1f3fdb28f0e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:43.832846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3af4f8-3ff8-4a7c-9609-1f2e045a894e None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:43.833139 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3af4f8-3ff8-4a7c-9609-1f2e045a894e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:43.833693 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3af4f8-3ff8-4a7c-9609-1f2e045a894e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:43.850579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 954/1877] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:43 2022] PUT /identity/v3/projects/54cbfe298d34475cbf86b5c0be99acda/users/2fbdcaa712b84abdba95a8eb6c8843da/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:43.852641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a54d5-0f93-4c1f-a271-2f5e6a980fee None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:43.852950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a54d5-0f93-4c1f-a271-2f5e6a980fee None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:43.853227 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a54d5-0f93-4c1f-a271-2f5e6a980fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:43.869849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-df3af4f8-3ff8-4a7c-9609-1f2e045a894e None None] MFA Rules not processed for user `8d7fe01ef64f46f6a0e77d5945219b33`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:43.872740 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-df3af4f8-3ff8-4a7c-9609-1f2e045a894e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:43.901009 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b58a54d5-0f93-4c1f-a271-2f5e6a980fee None None] MFA Rules not processed for user `2fbdcaa712b84abdba95a8eb6c8843da`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:43.904794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b58a54d5-0f93-4c1f-a271-2f5e6a980fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:44.007094 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 924/1878] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Jun 27 21:16:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:44.008028 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3af4f8-3ff8-4a7c-9609-1f2e045a894e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:44.014252 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-df3af4f8-3ff8-4a7c-9609-1f2e045a894e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:44.053781 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:44.065756 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 955/1879] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:44.079932 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a54d5-0f93-4c1f-a271-2f5e6a980fee None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:44.082845 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b58a54d5-0f93-4c1f-a271-2f5e6a980fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:44.103010 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a54d5-0f93-4c1f-a271-2f5e6a980fee None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:44.112666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69df57-76a8-49cb-a805-457489588292 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:44.113522 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69df57-76a8-49cb-a805-457489588292 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:44.113800 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69df57-76a8-49cb-a805-457489588292 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:44.114095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69df57-76a8-49cb-a805-457489588292 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:44.126356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69df57-76a8-49cb-a805-457489588292 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:44.128088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69df57-76a8-49cb-a805-457489588292 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:44.142563 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 956/1880] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:44.143927 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69df57-76a8-49cb-a805-457489588292 service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:44.145319 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ce69df57-76a8-49cb-a805-457489588292 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:44.174072 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3af4f8-3ff8-4a7c-9609-1f2e045a894e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:44.184389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137b0e1-52c7-43fb-9461-23ba89f7e608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:44.186950 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137b0e1-52c7-43fb-9461-23ba89f7e608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:44.187219 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137b0e1-52c7-43fb-9461-23ba89f7e608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:44.188667 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137b0e1-52c7-43fb-9461-23ba89f7e608 admin admin] PATH_INFO: `/v3/users/81f9256b810746e38b3d09fc2668e591` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:44.191543 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137b0e1-52c7-43fb-9461-23ba89f7e608 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81f9256b810746e38b3d09fc2668e591)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:44.195646 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137b0e1-52c7-43fb-9461-23ba89f7e608 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:44.254481 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69df57-76a8-49cb-a805-457489588292 service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:44.262744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d59abf-9f64-4f7a-91d5-6613f6bf8f2f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b43db1fd0b494a3f8588b498b585f247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:44.263440 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d59abf-9f64-4f7a-91d5-6613f6bf8f2f service glance] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:44.263844 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d59abf-9f64-4f7a-91d5-6613f6bf8f2f service glance] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:44.264089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d59abf-9f64-4f7a-91d5-6613f6bf8f2f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:44.284166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d59abf-9f64-4f7a-91d5-6613f6bf8f2f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:44.288573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d59abf-9f64-4f7a-91d5-6613f6bf8f2f service glance] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:44.309349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 957/1881] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:44.313356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6137b0e1-52c7-43fb-9461-23ba89f7e608 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81f9256b810746e38b3d09fc2668e591'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:44.314856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6137b0e1-52c7-43fb-9461-23ba89f7e608 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81f9256b810746e38b3d09fc2668e591'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:44.315708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6137b0e1-52c7-43fb-9461-23ba89f7e608 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81f9256b810746e38b3d09fc2668e591'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:44.330872 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6137b0e1-52c7-43fb-9461-23ba89f7e608 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81f9256b810746e38b3d09fc2668e591'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:44.339717 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6137b0e1-52c7-43fb-9461-23ba89f7e608 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81f9256b810746e38b3d09fc2668e591'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:44.365880 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 925/1882] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:44 2022] DELETE /identity/v3/users/81f9256b810746e38b3d09fc2668e591 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:44.370764 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c3375-edc7-473e-a134-0c3a2f65ce9d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:44.371050 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c3375-edc7-473e-a134-0c3a2f65ce9d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:44.371351 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c3375-edc7-473e-a134-0c3a2f65ce9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:44.409307 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-393c3375-edc7-473e-a134-0c3a2f65ce9d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:44.413096 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-393c3375-edc7-473e-a134-0c3a2f65ce9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:44.541995 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 958/1883] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:44.560593 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137b0e1-52c7-43fb-9461-23ba89f7e608 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:44.562208 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6137b0e1-52c7-43fb-9461-23ba89f7e608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:44.629372 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137b0e1-52c7-43fb-9461-23ba89f7e608 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:44.633501 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843b6cd-84cc-42af-9f77-ac5460e85f73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:44.634323 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843b6cd-84cc-42af-9f77-ac5460e85f73 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:44.634761 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843b6cd-84cc-42af-9f77-ac5460e85f73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:44.635178 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843b6cd-84cc-42af-9f77-ac5460e85f73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:44.646082 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843b6cd-84cc-42af-9f77-ac5460e85f73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:44.647663 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843b6cd-84cc-42af-9f77-ac5460e85f73 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:44.658473 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 926/1884] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:44.781302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c3375-edc7-473e-a134-0c3a2f65ce9d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:44.782763 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-393c3375-edc7-473e-a134-0c3a2f65ce9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:44.794680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:44.847879 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c3375-edc7-473e-a134-0c3a2f65ce9d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:44.872175 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bea44-7e5c-4d07-8764-c1c8ef7738f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:44.873172 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bea44-7e5c-4d07-8764-c1c8ef7738f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:44.873721 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bea44-7e5c-4d07-8764-c1c8ef7738f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:44.873828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bea44-7e5c-4d07-8764-c1c8ef7738f4 admin admin] PATH_INFO: `/v3/projects/828460b6b1e74475a73198375ede31cc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:44.876009 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7bea44-7e5c-4d07-8764-c1c8ef7738f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=828460b6b1e74475a73198375ede31cc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:44.878230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7bea44-7e5c-4d07-8764-c1c8ef7738f4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:44.962226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5b7bea44-7e5c-4d07-8764-c1c8ef7738f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '828460b6b1e74475a73198375ede31cc', 'request_id': 'req-5b7bea44-7e5c-4d07-8764-c1c8ef7738f4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:44.977628 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5b7bea44-7e5c-4d07-8764-c1c8ef7738f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '828460b6b1e74475a73198375ede31cc', 'request_id': 'req-5b7bea44-7e5c-4d07-8764-c1c8ef7738f4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:44.984849 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5b7bea44-7e5c-4d07-8764-c1c8ef7738f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '828460b6b1e74475a73198375ede31cc', 'request_id': 'req-5b7bea44-7e5c-4d07-8764-c1c8ef7738f4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.008335 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5b7bea44-7e5c-4d07-8764-c1c8ef7738f4 admin admin] The token cache is being invalidate because project 828460b6b1e74475a73198375ede31cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:45.009550 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5b7bea44-7e5c-4d07-8764-c1c8ef7738f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.010740 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5b7bea44-7e5c-4d07-8764-c1c8ef7738f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.013051 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 959/1885] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:44 2022] DELETE /identity/v3/projects/828460b6b1e74475a73198375ede31cc => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:45.017983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843b6cd-84cc-42af-9f77-ac5460e85f73 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:45.040929 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843b6cd-84cc-42af-9f77-ac5460e85f73 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:45.094348 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bea44-7e5c-4d07-8764-c1c8ef7738f4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:45.096755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5b7bea44-7e5c-4d07-8764-c1c8ef7738f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:45.098545 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:45.144051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f67dc-95ec-4cbd-ae94-3b2369829ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:45.145163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f67dc-95ec-4cbd-ae94-3b2369829ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:45.146078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f67dc-95ec-4cbd-ae94-3b2369829ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:45.146378 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f67dc-95ec-4cbd-ae94-3b2369829ca7 admin admin] PATH_INFO: `/v3/users/6c2caa953829470eb5aaffa885e9970c` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:45.147696 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f67dc-95ec-4cbd-ae94-3b2369829ca7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c2caa953829470eb5aaffa885e9970c)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:45.150425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f67dc-95ec-4cbd-ae94-3b2369829ca7 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:45.170972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bea44-7e5c-4d07-8764-c1c8ef7738f4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:45.176762 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200abf02-d71a-4d36-a419-0e9892f2e23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:45.178456 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200abf02-d71a-4d36-a419-0e9892f2e23c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:45.178456 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200abf02-d71a-4d36-a419-0e9892f2e23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:45.178735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200abf02-d71a-4d36-a419-0e9892f2e23c admin admin] PATH_INFO: `/v3/projects/807012405c0a4f659a06a35c0cc342cb` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:45.180246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200abf02-d71a-4d36-a419-0e9892f2e23c admin admin] RBAC: Authorizing `identity:delete_project(project_id=807012405c0a4f659a06a35c0cc342cb)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:45.182321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200abf02-d71a-4d36-a419-0e9892f2e23c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:45.205406 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8e6f67dc-95ec-4cbd-ae94-3b2369829ca7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c2caa953829470eb5aaffa885e9970c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.206138 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8e6f67dc-95ec-4cbd-ae94-3b2369829ca7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c2caa953829470eb5aaffa885e9970c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.206805 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8e6f67dc-95ec-4cbd-ae94-3b2369829ca7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c2caa953829470eb5aaffa885e9970c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.220797 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8e6f67dc-95ec-4cbd-ae94-3b2369829ca7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c2caa953829470eb5aaffa885e9970c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.227894 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8e6f67dc-95ec-4cbd-ae94-3b2369829ca7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c2caa953829470eb5aaffa885e9970c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.240648 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-200abf02-d71a-4d36-a419-0e9892f2e23c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '807012405c0a4f659a06a35c0cc342cb', 'request_id': 'req-200abf02-d71a-4d36-a419-0e9892f2e23c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.246703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 927/1886] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:45 2022] DELETE /identity/v3/users/6c2caa953829470eb5aaffa885e9970c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:45.249719 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-200abf02-d71a-4d36-a419-0e9892f2e23c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '807012405c0a4f659a06a35c0cc342cb', 'request_id': 'req-200abf02-d71a-4d36-a419-0e9892f2e23c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.254491 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-200abf02-d71a-4d36-a419-0e9892f2e23c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '807012405c0a4f659a06a35c0cc342cb', 'request_id': 'req-200abf02-d71a-4d36-a419-0e9892f2e23c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.266238 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-200abf02-d71a-4d36-a419-0e9892f2e23c admin admin] The token cache is being invalidate because project 807012405c0a4f659a06a35c0cc342cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:45.266528 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-200abf02-d71a-4d36-a419-0e9892f2e23c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.267198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-200abf02-d71a-4d36-a419-0e9892f2e23c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.269353 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 960/1887] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:45 2022] DELETE /identity/v3/projects/807012405c0a4f659a06a35c0cc342cb => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:45.273113 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f67dc-95ec-4cbd-ae94-3b2369829ca7 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:45.274686 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8e6f67dc-95ec-4cbd-ae94-3b2369829ca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:45.346784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f67dc-95ec-4cbd-ae94-3b2369829ca7 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:45.352601 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f7e044-850a-4256-896d-3b7084d7c1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:45.353644 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7e044-850a-4256-896d-3b7084d7c1c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:45.353988 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7e044-850a-4256-896d-3b7084d7c1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:45.354334 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7e044-850a-4256-896d-3b7084d7c1c1 admin admin] PATH_INFO: `/v3/users/41c4f6977f30483089610842c2865fdf` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:45.355736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f7e044-850a-4256-896d-3b7084d7c1c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c4f6977f30483089610842c2865fdf)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:45.357950 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f7e044-850a-4256-896d-3b7084d7c1c1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:45.419280 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f3f7e044-850a-4256-896d-3b7084d7c1c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c4f6977f30483089610842c2865fdf'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.420129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f3f7e044-850a-4256-896d-3b7084d7c1c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c4f6977f30483089610842c2865fdf'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.420862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f3f7e044-850a-4256-896d-3b7084d7c1c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c4f6977f30483089610842c2865fdf'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.428970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:45.436186 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f3f7e044-850a-4256-896d-3b7084d7c1c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c4f6977f30483089610842c2865fdf'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.445117 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f3f7e044-850a-4256-896d-3b7084d7c1c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c4f6977f30483089610842c2865fdf'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.458523 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 928/1888] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:45 2022] DELETE /identity/v3/users/41c4f6977f30483089610842c2865fdf => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:45.566789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200abf02-d71a-4d36-a419-0e9892f2e23c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:45.568418 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-200abf02-d71a-4d36-a419-0e9892f2e23c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:45.599467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:45.662660 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200abf02-d71a-4d36-a419-0e9892f2e23c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:45.737044 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e81e2b-0f49-4bc8-99da-f25dc2da5f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:45.738327 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e81e2b-0f49-4bc8-99da-f25dc2da5f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:45.738753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e81e2b-0f49-4bc8-99da-f25dc2da5f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:45.739143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e81e2b-0f49-4bc8-99da-f25dc2da5f7f admin admin] PATH_INFO: `/v3/projects/303853dbfc2a4f1a8165c9abd3a031f2` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:45.740907 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e81e2b-0f49-4bc8-99da-f25dc2da5f7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=303853dbfc2a4f1a8165c9abd3a031f2)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:45.743114 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e81e2b-0f49-4bc8-99da-f25dc2da5f7f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:45.795857 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f7e044-850a-4256-896d-3b7084d7c1c1 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:45.797534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f3f7e044-850a-4256-896d-3b7084d7c1c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:45.817947 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f7e044-850a-4256-896d-3b7084d7c1c1 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:45.828965 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-16e81e2b-0f49-4bc8-99da-f25dc2da5f7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '303853dbfc2a4f1a8165c9abd3a031f2', 'request_id': 'req-16e81e2b-0f49-4bc8-99da-f25dc2da5f7f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.841313 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-16e81e2b-0f49-4bc8-99da-f25dc2da5f7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '303853dbfc2a4f1a8165c9abd3a031f2', 'request_id': 'req-16e81e2b-0f49-4bc8-99da-f25dc2da5f7f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.849138 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-16e81e2b-0f49-4bc8-99da-f25dc2da5f7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '303853dbfc2a4f1a8165c9abd3a031f2', 'request_id': 'req-16e81e2b-0f49-4bc8-99da-f25dc2da5f7f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.861931 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-16e81e2b-0f49-4bc8-99da-f25dc2da5f7f admin admin] The token cache is being invalidate because project 303853dbfc2a4f1a8165c9abd3a031f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:45.862467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-16e81e2b-0f49-4bc8-99da-f25dc2da5f7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.863269 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-16e81e2b-0f49-4bc8-99da-f25dc2da5f7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.865464 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 961/1889] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:45 2022] DELETE /identity/v3/projects/303853dbfc2a4f1a8165c9abd3a031f2 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:45.873095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20446f11-4ae9-484a-9678-8d7df41f3d79 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:45.873451 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20446f11-4ae9-484a-9678-8d7df41f3d79 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:45.873789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20446f11-4ae9-484a-9678-8d7df41f3d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:45.892046 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcebcf7-b31f-44f8-a9ed-9912dffb6b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:45.893250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcebcf7-b31f-44f8-a9ed-9912dffb6b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:45.893535 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcebcf7-b31f-44f8-a9ed-9912dffb6b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:45.893812 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcebcf7-b31f-44f8-a9ed-9912dffb6b4d admin admin] PATH_INFO: `/v3/projects/f4a9c77596a94deda399342c53b22092` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:45.894614 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-20446f11-4ae9-484a-9678-8d7df41f3d79 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:45.895024 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcebcf7-b31f-44f8-a9ed-9912dffb6b4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4a9c77596a94deda399342c53b22092)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:45.895873 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-20446f11-4ae9-484a-9678-8d7df41f3d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:45.897252 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcebcf7-b31f-44f8-a9ed-9912dffb6b4d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:45.911080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 962/1890] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:45.912543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb125d-98a7-4cd4-9fee-ed03c5fcc2ab None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:45.912984 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb125d-98a7-4cd4-9fee-ed03c5fcc2ab None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:45.913613 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb125d-98a7-4cd4-9fee-ed03c5fcc2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:45.933307 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a6cb125d-98a7-4cd4-9fee-ed03c5fcc2ab None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:45.934715 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a6cb125d-98a7-4cd4-9fee-ed03c5fcc2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:45.947431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ddcebcf7-b31f-44f8-a9ed-9912dffb6b4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4a9c77596a94deda399342c53b22092', 'request_id': 'req-ddcebcf7-b31f-44f8-a9ed-9912dffb6b4d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.956960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ddcebcf7-b31f-44f8-a9ed-9912dffb6b4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4a9c77596a94deda399342c53b22092', 'request_id': 'req-ddcebcf7-b31f-44f8-a9ed-9912dffb6b4d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.962840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ddcebcf7-b31f-44f8-a9ed-9912dffb6b4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4a9c77596a94deda399342c53b22092', 'request_id': 'req-ddcebcf7-b31f-44f8-a9ed-9912dffb6b4d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.975450 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ddcebcf7-b31f-44f8-a9ed-9912dffb6b4d admin admin] The token cache is being invalidate because project f4a9c77596a94deda399342c53b22092 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:45.975896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ddcebcf7-b31f-44f8-a9ed-9912dffb6b4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.977041 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ddcebcf7-b31f-44f8-a9ed-9912dffb6b4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:45.978956 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 929/1891] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:45 2022] DELETE /identity/v3/projects/f4a9c77596a94deda399342c53b22092 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:45.983750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f0e2d-78af-4efd-95fb-2cc766609146 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:45.984135 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f0e2d-78af-4efd-95fb-2cc766609146 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:45.984511 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f0e2d-78af-4efd-95fb-2cc766609146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:45.988306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:46.007787 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-893f0e2d-78af-4efd-95fb-2cc766609146 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:46.020453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-893f0e2d-78af-4efd-95fb-2cc766609146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:46.020558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 963/1892] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:46.020654 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 930/1893] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:46.020745 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb125d-98a7-4cd4-9fee-ed03c5fcc2ab None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.025414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ca5a8-df90-4ae4-93c4-ed722416cc97 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.025792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ca5a8-df90-4ae4-93c4-ed722416cc97 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.026066 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ca5a8-df90-4ae4-93c4-ed722416cc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.029897 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb125d-98a7-4cd4-9fee-ed03c5fcc2ab None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.036044 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7137f-2b22-4cad-aa21-5b58759af72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.036716 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7137f-2b22-4cad-aa21-5b58759af72c admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.037009 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7137f-2b22-4cad-aa21-5b58759af72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.037290 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7137f-2b22-4cad-aa21-5b58759af72c admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.038020 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7137f-2b22-4cad-aa21-5b58759af72c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:46.038334 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7137f-2b22-4cad-aa21-5b58759af72c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.039988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7137f-2b22-4cad-aa21-5b58759af72c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.043950 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-aa3ca5a8-df90-4ae4-93c4-ed722416cc97 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:46.045241 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-aa3ca5a8-df90-4ae4-93c4-ed722416cc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:46.052123 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 964/1894] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:16:46.053347 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 931/1895] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:46.056284 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7137f-2b22-4cad-aa21-5b58759af72c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.057450 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ca5a8-df90-4ae4-93c4-ed722416cc97 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.066207 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7137f-2b22-4cad-aa21-5b58759af72c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.066754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ca5a8-df90-4ae4-93c4-ed722416cc97 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.070945 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a31d8-99a5-4ebd-9ce8-e8b36fa24c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.071612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a31d8-99a5-4ebd-9ce8-e8b36fa24c34 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.071921 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a31d8-99a5-4ebd-9ce8-e8b36fa24c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.072200 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a31d8-99a5-4ebd-9ce8-e8b36fa24c34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.072887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a31d8-99a5-4ebd-9ce8-e8b36fa24c34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.073226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ca984-aa49-4c32-835a-36795257b938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.074333 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ca984-aa49-4c32-835a-36795257b938 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.074624 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ca984-aa49-4c32-835a-36795257b938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.074817 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a31d8-99a5-4ebd-9ce8-e8b36fa24c34 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.074918 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ca984-aa49-4c32-835a-36795257b938 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.075644 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ca984-aa49-4c32-835a-36795257b938 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:46.075998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ca984-aa49-4c32-835a-36795257b938 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.078359 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ca984-aa49-4c32-835a-36795257b938 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.094433 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 932/1896] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:16:46.097228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ca984-aa49-4c32-835a-36795257b938 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.108080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ca984-aa49-4c32-835a-36795257b938 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.110467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 965/1897] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:46 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:46.114414 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a31d8-99a5-4ebd-9ce8-e8b36fa24c34 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.147115 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:46.180773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e08134-d639-473b-b0ac-ebb9d2d641bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.181600 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e08134-d639-473b-b0ac-ebb9d2d641bd admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.181951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e08134-d639-473b-b0ac-ebb9d2d641bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.182301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e08134-d639-473b-b0ac-ebb9d2d641bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.183073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e08134-d639-473b-b0ac-ebb9d2d641bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.185193 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e08134-d639-473b-b0ac-ebb9d2d641bd admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.195328 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a31d8-99a5-4ebd-9ce8-e8b36fa24c34 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.200808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0ef6d-3be9-46c4-812e-d84fca38fde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.201582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0ef6d-3be9-46c4-812e-d84fca38fde5 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.201891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0ef6d-3be9-46c4-812e-d84fca38fde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.202177 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0ef6d-3be9-46c4-812e-d84fca38fde5 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.202912 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0ef6d-3be9-46c4-812e-d84fca38fde5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.205023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0ef6d-3be9-46c4-812e-d84fca38fde5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.214255 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 933/1898] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:46 2022] POST /identity/v3/projects => generated 389 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:46.218613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e08134-d639-473b-b0ac-ebb9d2d641bd admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.226915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 966/1899] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:46 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:46.231231 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0ef6d-3be9-46c4-812e-d84fca38fde5 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.241189 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0ef6d-3be9-46c4-812e-d84fca38fde5 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.307915 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e08134-d639-473b-b0ac-ebb9d2d641bd admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.311996 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924b817-6c43-4b35-8c76-31dd27099e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.312808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924b817-6c43-4b35-8c76-31dd27099e6f admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.312921 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880a8f6-4f29-44b6-8c3d-5c2333c529f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.313291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924b817-6c43-4b35-8c76-31dd27099e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.313566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880a8f6-4f29-44b6-8c3d-5c2333c529f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.313844 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880a8f6-4f29-44b6-8c3d-5c2333c529f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.313948 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924b817-6c43-4b35-8c76-31dd27099e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.314177 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880a8f6-4f29-44b6-8c3d-5c2333c529f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.314919 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d880a8f6-4f29-44b6-8c3d-5c2333c529f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.315080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b924b817-6c43-4b35-8c76-31dd27099e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.316888 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b924b817-6c43-4b35-8c76-31dd27099e6f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.317394 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d880a8f6-4f29-44b6-8c3d-5c2333c529f1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.328253 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 967/1900] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:16:46.332192 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924b817-6c43-4b35-8c76-31dd27099e6f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.339103 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 934/1901] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:46 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:46.342531 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924b817-6c43-4b35-8c76-31dd27099e6f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.343128 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880a8f6-4f29-44b6-8c3d-5c2333c529f1 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.347243 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a4bae-323e-481c-ad56-0eef41993daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.347941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a4bae-323e-481c-ad56-0eef41993daf admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.348230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a4bae-323e-481c-ad56-0eef41993daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.348559 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a4bae-323e-481c-ad56-0eef41993daf admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.349222 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a4bae-323e-481c-ad56-0eef41993daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.350831 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a4bae-323e-481c-ad56-0eef41993daf admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.353834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880a8f6-4f29-44b6-8c3d-5c2333c529f1 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.358852 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5abc8-0998-49d6-9062-bc3d214234aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.359502 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5abc8-0998-49d6-9062-bc3d214234aa admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.359818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5abc8-0998-49d6-9062-bc3d214234aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.360094 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5abc8-0998-49d6-9062-bc3d214234aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.360808 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5abc8-0998-49d6-9062-bc3d214234aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.362271 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 968/1902] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:16:46.362367 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5abc8-0998-49d6-9062-bc3d214234aa admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.366547 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a4bae-323e-481c-ad56-0eef41993daf admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.373219 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 935/1903] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:16:46.376336 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a4bae-323e-481c-ad56-0eef41993daf admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.377095 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5abc8-0998-49d6-9062-bc3d214234aa admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.380908 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0e5b1-18e3-40ee-8544-8672738113a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.381844 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0e5b1-18e3-40ee-8544-8672738113a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.382297 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0e5b1-18e3-40ee-8544-8672738113a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.382575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0e5b1-18e3-40ee-8544-8672738113a3 admin admin] PATH_INFO: `/v3/projects/e94dc9bb6911462192ee42f61a1cc784/users/f7c13026ffbb48dfbebf9751fbb0dbc1/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.386932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5abc8-0998-49d6-9062-bc3d214234aa admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.391591 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9b54e-621e-4568-ba57-50749b1d0a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.392263 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9b54e-621e-4568-ba57-50749b1d0a0c admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.392547 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9b54e-621e-4568-ba57-50749b1d0a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.393138 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9b54e-621e-4568-ba57-50749b1d0a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.393880 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9b54e-621e-4568-ba57-50749b1d0a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.395513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9b54e-621e-4568-ba57-50749b1d0a0c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.398146 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb0e5b1-18e3-40ee-8544-8672738113a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e94dc9bb6911462192ee42f61a1cc784, user_id=f7c13026ffbb48dfbebf9751fbb0dbc1, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.400843 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb0e5b1-18e3-40ee-8544-8672738113a3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.406231 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 936/1904] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:16:46.410621 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9b54e-621e-4568-ba57-50749b1d0a0c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.415543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 969/1905] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:46 2022] PUT /identity/v3/projects/e94dc9bb6911462192ee42f61a1cc784/users/f7c13026ffbb48dfbebf9751fbb0dbc1/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:46.421029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9b54e-621e-4568-ba57-50749b1d0a0c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.426308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efc03c-5f9b-429f-8434-42d8e15b2242 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.426583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efc03c-5f9b-429f-8434-42d8e15b2242 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.426867 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efc03c-5f9b-429f-8434-42d8e15b2242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.433313 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:46.453995 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-59efc03c-5f9b-429f-8434-42d8e15b2242 None None] MFA Rules not processed for user `f7c13026ffbb48dfbebf9751fbb0dbc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:46.455223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-59efc03c-5f9b-429f-8434-42d8e15b2242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:46.478359 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6232b7-0783-4876-a67b-107096b8b153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.479339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6232b7-0783-4876-a67b-107096b8b153 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.479711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6232b7-0783-4876-a67b-107096b8b153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.480125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6232b7-0783-4876-a67b-107096b8b153 admin admin] PATH_INFO: `/v3/projects/e658ba7a93cd4042b45d510a1dfc7005/users/173cbc08cf5041938ab05fb8271a49c4/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.494168 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6232b7-0783-4876-a67b-107096b8b153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e658ba7a93cd4042b45d510a1dfc7005, user_id=173cbc08cf5041938ab05fb8271a49c4, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.496830 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6232b7-0783-4876-a67b-107096b8b153 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.510521 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 937/1906] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:46 2022] PUT /identity/v3/projects/e658ba7a93cd4042b45d510a1dfc7005/users/173cbc08cf5041938ab05fb8271a49c4/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:46.520532 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 970/1907] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:46.521191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847eb58-425f-45d7-b5cf-f21ad86ba28b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.521462 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847eb58-425f-45d7-b5cf-f21ad86ba28b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.521746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847eb58-425f-45d7-b5cf-f21ad86ba28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.524666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efc03c-5f9b-429f-8434-42d8e15b2242 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.534896 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efc03c-5f9b-429f-8434-42d8e15b2242 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.546925 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:46.550347 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e847eb58-425f-45d7-b5cf-f21ad86ba28b None None] MFA Rules not processed for user `173cbc08cf5041938ab05fb8271a49c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:46.551815 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e847eb58-425f-45d7-b5cf-f21ad86ba28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:46.591235 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed43c7-3e4a-4c62-a75a-c43fd84479bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.591969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed43c7-3e4a-4c62-a75a-c43fd84479bd admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.592249 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed43c7-3e4a-4c62-a75a-c43fd84479bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.592527 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed43c7-3e4a-4c62-a75a-c43fd84479bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.593212 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed43c7-3e4a-4c62-a75a-c43fd84479bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.595121 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed43c7-3e4a-4c62-a75a-c43fd84479bd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.617035 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 938/1908] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:46.621138 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 971/1909] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:46 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:46.621278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847eb58-425f-45d7-b5cf-f21ad86ba28b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.624907 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed43c7-3e4a-4c62-a75a-c43fd84479bd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.631758 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847eb58-425f-45d7-b5cf-f21ad86ba28b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.686093 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed43c7-3e4a-4c62-a75a-c43fd84479bd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.688456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3fdeb5-cdae-4b8e-8449-0621823d6b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.689115 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3fdeb5-cdae-4b8e-8449-0621823d6b24 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.689400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3fdeb5-cdae-4b8e-8449-0621823d6b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.689675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3fdeb5-cdae-4b8e-8449-0621823d6b24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.690365 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3fdeb5-cdae-4b8e-8449-0621823d6b24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.691190 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df839f-f974-4adb-8892-e234a4b6f56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.691925 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df839f-f974-4adb-8892-e234a4b6f56e admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.692207 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df839f-f974-4adb-8892-e234a4b6f56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.692308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3fdeb5-cdae-4b8e-8449-0621823d6b24 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.692478 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df839f-f974-4adb-8892-e234a4b6f56e admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.693171 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df839f-f974-4adb-8892-e234a4b6f56e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.695064 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df839f-f974-4adb-8892-e234a4b6f56e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.715968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 972/1910] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:46 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:46.719864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df839f-f974-4adb-8892-e234a4b6f56e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.722486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 939/1911] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:46 2022] POST /identity/v3/projects => generated 389 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:46.725460 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3fdeb5-cdae-4b8e-8449-0621823d6b24 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.730055 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df839f-f974-4adb-8892-e234a4b6f56e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.734968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:46.787063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3fdeb5-cdae-4b8e-8449-0621823d6b24 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.789107 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6516cae7-884c-4807-9dc5-f1b17d1c9987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.789797 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516cae7-884c-4807-9dc5-f1b17d1c9987 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.790188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516cae7-884c-4807-9dc5-f1b17d1c9987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.792089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516cae7-884c-4807-9dc5-f1b17d1c9987 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.792886 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3af210-c216-419a-a6dc-4bb892f9b2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.793018 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6516cae7-884c-4807-9dc5-f1b17d1c9987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.793726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3af210-c216-419a-a6dc-4bb892f9b2c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.794079 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3af210-c216-419a-a6dc-4bb892f9b2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.794422 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3af210-c216-419a-a6dc-4bb892f9b2c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.794996 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6516cae7-884c-4807-9dc5-f1b17d1c9987 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.795311 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3af210-c216-419a-a6dc-4bb892f9b2c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.798224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3af210-c216-419a-a6dc-4bb892f9b2c1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.810974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 973/1912] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:46.813010 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6516cae7-884c-4807-9dc5-f1b17d1c9987 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.824768 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6516cae7-884c-4807-9dc5-f1b17d1c9987 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.827676 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 940/1913] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:46 2022] POST /identity/v3/users => generated 301 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:46.830345 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3af210-c216-419a-a6dc-4bb892f9b2c1 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.830466 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246fd2c-fc76-403a-ad5e-6ed259f8458e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.831327 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246fd2c-fc76-403a-ad5e-6ed259f8458e admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.831633 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246fd2c-fc76-403a-ad5e-6ed259f8458e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.832160 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246fd2c-fc76-403a-ad5e-6ed259f8458e admin admin] PATH_INFO: `/v3/projects/406c455d9dfa4804bf14f8a27f1182f3/users/1730eaa01d894396986343ef7b788779/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.841353 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3af210-c216-419a-a6dc-4bb892f9b2c1 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.846535 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4563d0-45d5-4d3e-bfdf-4fdae262ba22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.847219 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4563d0-45d5-4d3e-bfdf-4fdae262ba22 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.847505 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4563d0-45d5-4d3e-bfdf-4fdae262ba22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.847810 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4563d0-45d5-4d3e-bfdf-4fdae262ba22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.848522 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4563d0-45d5-4d3e-bfdf-4fdae262ba22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.848985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246fd2c-fc76-403a-ad5e-6ed259f8458e admin admin] RBAC: Authorizing `identity:create_grant(project_id=406c455d9dfa4804bf14f8a27f1182f3, user_id=1730eaa01d894396986343ef7b788779, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.850136 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4563d0-45d5-4d3e-bfdf-4fdae262ba22 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.852469 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246fd2c-fc76-403a-ad5e-6ed259f8458e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.861707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 941/1914] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:16:46.865968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4563d0-45d5-4d3e-bfdf-4fdae262ba22 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.869322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 974/1915] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:46 2022] PUT /identity/v3/projects/406c455d9dfa4804bf14f8a27f1182f3/users/1730eaa01d894396986343ef7b788779/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:46.873063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246fd2c-fc76-403a-ad5e-6ed259f8458e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.877078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4563d0-45d5-4d3e-bfdf-4fdae262ba22 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.881893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:46.942039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246fd2c-fc76-403a-ad5e-6ed259f8458e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.948640 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f2759-7590-4885-a15c-69acf91ccc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.949343 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f2759-7590-4885-a15c-69acf91ccc5b admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.949630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f2759-7590-4885-a15c-69acf91ccc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.949910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f2759-7590-4885-a15c-69acf91ccc5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.950643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f2759-7590-4885-a15c-69acf91ccc5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.952335 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f2759-7590-4885-a15c-69acf91ccc5b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.963210 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aca055-c660-4bf0-ac30-eb3bca731d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.964233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aca055-c660-4bf0-ac30-eb3bca731d44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.967941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 975/1916] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:16:46.968092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aca055-c660-4bf0-ac30-eb3bca731d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.968092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aca055-c660-4bf0-ac30-eb3bca731d44 admin admin] PATH_INFO: `/v3/projects/b3cc9379a2114b03b195fedd6969120b/users/24380b69aa9f43d68a44dc810cdd707d/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.969520 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f2759-7590-4885-a15c-69acf91ccc5b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.982904 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f2759-7590-4885-a15c-69acf91ccc5b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.987835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b42b9-4f39-4fa1-8f7c-5031cde1d849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.993742 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b42b9-4f39-4fa1-8f7c-5031cde1d849 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.993912 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aca055-c660-4bf0-ac30-eb3bca731d44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3cc9379a2114b03b195fedd6969120b, user_id=24380b69aa9f43d68a44dc810cdd707d, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.994016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b42b9-4f39-4fa1-8f7c-5031cde1d849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.994016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b42b9-4f39-4fa1-8f7c-5031cde1d849 admin admin] PATH_INFO: `/v3/projects/406c455d9dfa4804bf14f8a27f1182f3/users/1730eaa01d894396986343ef7b788779/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.994016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b42b9-4f39-4fa1-8f7c-5031cde1d849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=406c455d9dfa4804bf14f8a27f1182f3, user_id=1730eaa01d894396986343ef7b788779, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.994260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aca055-c660-4bf0-ac30-eb3bca731d44 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.994819 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b42b9-4f39-4fa1-8f7c-5031cde1d849 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:47.008904 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 942/1917] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:46 2022] PUT /identity/v3/projects/b3cc9379a2114b03b195fedd6969120b/users/24380b69aa9f43d68a44dc810cdd707d/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:47.009416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 976/1918] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:46 2022] PUT /identity/v3/projects/406c455d9dfa4804bf14f8a27f1182f3/users/1730eaa01d894396986343ef7b788779/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:47.013004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aca055-c660-4bf0-ac30-eb3bca731d44 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:47.019691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7adff-3ad8-4f45-9b14-fd2c16ab91b0 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:47.020232 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7adff-3ad8-4f45-9b14-fd2c16ab91b0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:47.020584 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7adff-3ad8-4f45-9b14-fd2c16ab91b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:47.054985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b4e7adff-3ad8-4f45-9b14-fd2c16ab91b0 None None] MFA Rules not processed for user `1730eaa01d894396986343ef7b788779`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:47.056757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b4e7adff-3ad8-4f45-9b14-fd2c16ab91b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:47.093329 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aca055-c660-4bf0-ac30-eb3bca731d44 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:47.098730 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8527cd-1f60-4cf2-a8ff-f31f1d471d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:47.099526 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8527cd-1f60-4cf2-a8ff-f31f1d471d4e admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:47.099933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8527cd-1f60-4cf2-a8ff-f31f1d471d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:47.100312 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8527cd-1f60-4cf2-a8ff-f31f1d471d4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:47.101144 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8527cd-1f60-4cf2-a8ff-f31f1d471d4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:47.103325 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8527cd-1f60-4cf2-a8ff-f31f1d471d4e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:47.114373 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 943/1919] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:47.118885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8527cd-1f60-4cf2-a8ff-f31f1d471d4e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:47.124648 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:47.134235 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8527cd-1f60-4cf2-a8ff-f31f1d471d4e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:47.139089 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14cb21-639f-47ae-bf1f-ac638fa4e199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:47.140083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14cb21-639f-47ae-bf1f-ac638fa4e199 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:47.140465 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14cb21-639f-47ae-bf1f-ac638fa4e199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:47.140816 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14cb21-639f-47ae-bf1f-ac638fa4e199 admin admin] PATH_INFO: `/v3/projects/b3cc9379a2114b03b195fedd6969120b/users/24380b69aa9f43d68a44dc810cdd707d/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:47.141205 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 977/1920] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:47 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:47.143244 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14cb21-639f-47ae-bf1f-ac638fa4e199 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3cc9379a2114b03b195fedd6969120b, user_id=24380b69aa9f43d68a44dc810cdd707d, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:47.146077 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14cb21-639f-47ae-bf1f-ac638fa4e199 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:47.152254 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7adff-3ad8-4f45-9b14-fd2c16ab91b0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:47.153620 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b4e7adff-3ad8-4f45-9b14-fd2c16ab91b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:47.156953 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 944/1921] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:47 2022] PUT /identity/v3/projects/b3cc9379a2114b03b195fedd6969120b/users/24380b69aa9f43d68a44dc810cdd707d/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:47.168006 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5d62d-8747-4800-9f5f-3df7b86a660d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:47.168298 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5d62d-8747-4800-9f5f-3df7b86a660d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:47.168578 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5d62d-8747-4800-9f5f-3df7b86a660d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:47.194641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ebd5d62d-8747-4800-9f5f-3df7b86a660d None None] MFA Rules not processed for user `24380b69aa9f43d68a44dc810cdd707d`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:47.195897 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ebd5d62d-8747-4800-9f5f-3df7b86a660d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:47.211382 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7adff-3ad8-4f45-9b14-fd2c16ab91b0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:47.258184 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1cc4a-4d34-4acb-8e73-be07525a0f59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:47.258831 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1cc4a-4d34-4acb-8e73-be07525a0f59 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:47.259111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1cc4a-4d34-4acb-8e73-be07525a0f59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:47.259386 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1cc4a-4d34-4acb-8e73-be07525a0f59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:47.260714 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:47.270096 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 945/1922] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:47.282131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5d62d-8747-4800-9f5f-3df7b86a660d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:47.283635 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ebd5d62d-8747-4800-9f5f-3df7b86a660d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:47.303757 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5d62d-8747-4800-9f5f-3df7b86a660d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:47.314296 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc158416-8bad-4ad9-bbee-6a7a49686682 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:47.314943 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc158416-8bad-4ad9-bbee-6a7a49686682 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:47.315224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc158416-8bad-4ad9-bbee-6a7a49686682 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:47.315495 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc158416-8bad-4ad9-bbee-6a7a49686682 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:47.333995 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc158416-8bad-4ad9-bbee-6a7a49686682 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:47.335983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc158416-8bad-4ad9-bbee-6a7a49686682 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:47.350410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 946/1923] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:47 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:47.372268 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1cc4a-4d34-4acb-8e73-be07525a0f59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:47.381067 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1cc4a-4d34-4acb-8e73-be07525a0f59 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:47.396748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 978/1924] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:16:47.873543 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc158416-8bad-4ad9-bbee-6a7a49686682 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:47.953428 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc158416-8bad-4ad9-bbee-6a7a49686682 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:47.958594 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6642190-9dc0-4a85-80c0-158653f30ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:47.959560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6642190-9dc0-4a85-80c0-158653f30ba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:47.959894 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6642190-9dc0-4a85-80c0-158653f30ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:47.960149 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6642190-9dc0-4a85-80c0-158653f30ba3 admin admin] PATH_INFO: `/v3/users/173cbc08cf5041938ab05fb8271a49c4` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:47.961534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6642190-9dc0-4a85-80c0-158653f30ba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=173cbc08cf5041938ab05fb8271a49c4)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:47.963662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6642190-9dc0-4a85-80c0-158653f30ba3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:48.028857 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c6642190-9dc0-4a85-80c0-158653f30ba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '173cbc08cf5041938ab05fb8271a49c4'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:48.029718 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c6642190-9dc0-4a85-80c0-158653f30ba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '173cbc08cf5041938ab05fb8271a49c4'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:48.030446 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c6642190-9dc0-4a85-80c0-158653f30ba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '173cbc08cf5041938ab05fb8271a49c4'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:48.044176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c6642190-9dc0-4a85-80c0-158653f30ba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '173cbc08cf5041938ab05fb8271a49c4'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:48.060410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c6642190-9dc0-4a85-80c0-158653f30ba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '173cbc08cf5041938ab05fb8271a49c4'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:48.083850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 947/1925] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:47 2022] DELETE /identity/v3/users/173cbc08cf5041938ab05fb8271a49c4 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:48.089505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c173afa-91f8-4c1e-a65f-ae36cdfc1b07 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:48.089822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c173afa-91f8-4c1e-a65f-ae36cdfc1b07 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:48.090103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c173afa-91f8-4c1e-a65f-ae36cdfc1b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:48.120994 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4c173afa-91f8-4c1e-a65f-ae36cdfc1b07 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:48.124041 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4c173afa-91f8-4c1e-a65f-ae36cdfc1b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:48.221141 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:48.238164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 979/1926] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:48.248682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6642190-9dc0-4a85-80c0-158653f30ba3 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:48.250469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c6642190-9dc0-4a85-80c0-158653f30ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:48.271661 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c173afa-91f8-4c1e-a65f-ae36cdfc1b07 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:48.273101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4c173afa-91f8-4c1e-a65f-ae36cdfc1b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:48.303649 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c173afa-91f8-4c1e-a65f-ae36cdfc1b07 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:48.326127 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa27fe2-adce-4e1b-b94b-bfd3e81675aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:48.327287 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa27fe2-adce-4e1b-b94b-bfd3e81675aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:48.332235 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa27fe2-adce-4e1b-b94b-bfd3e81675aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:48.332694 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa27fe2-adce-4e1b-b94b-bfd3e81675aa admin admin] PATH_INFO: `/v3/users/f7c13026ffbb48dfbebf9751fbb0dbc1` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:48.336246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa27fe2-adce-4e1b-b94b-bfd3e81675aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7c13026ffbb48dfbebf9751fbb0dbc1)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:48.338331 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa27fe2-adce-4e1b-b94b-bfd3e81675aa admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:48.352163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6642190-9dc0-4a85-80c0-158653f30ba3 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:48.356805 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e5009-6c90-4261-a993-1521ad904ee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:48.357701 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e5009-6c90-4261-a993-1521ad904ee7 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:48.358058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e5009-6c90-4261-a993-1521ad904ee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:48.358406 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e5009-6c90-4261-a993-1521ad904ee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:48.368946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e5009-6c90-4261-a993-1521ad904ee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:48.370610 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e5009-6c90-4261-a993-1521ad904ee7 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:48.386194 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 948/1927] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:48.406319 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8fa27fe2-adce-4e1b-b94b-bfd3e81675aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7c13026ffbb48dfbebf9751fbb0dbc1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:48.407327 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8fa27fe2-adce-4e1b-b94b-bfd3e81675aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7c13026ffbb48dfbebf9751fbb0dbc1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:48.408291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8fa27fe2-adce-4e1b-b94b-bfd3e81675aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7c13026ffbb48dfbebf9751fbb0dbc1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:48.423468 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8fa27fe2-adce-4e1b-b94b-bfd3e81675aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7c13026ffbb48dfbebf9751fbb0dbc1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:48.432227 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8fa27fe2-adce-4e1b-b94b-bfd3e81675aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7c13026ffbb48dfbebf9751fbb0dbc1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:48.449519 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 980/1928] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:48 2022] DELETE /identity/v3/users/f7c13026ffbb48dfbebf9751fbb0dbc1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:48.455431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a98b83-6f94-4b40-ae21-9fd923d4248d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:48.456002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a98b83-6f94-4b40-ae21-9fd923d4248d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:48.457295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a98b83-6f94-4b40-ae21-9fd923d4248d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:48.473047 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:48.481075 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-40a98b83-6f94-4b40-ae21-9fd923d4248d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:48.482424 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-40a98b83-6f94-4b40-ae21-9fd923d4248d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:48.592875 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 949/1929] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:48.605649 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa27fe2-adce-4e1b-b94b-bfd3e81675aa admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:48.607817 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8fa27fe2-adce-4e1b-b94b-bfd3e81675aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:48.730709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa27fe2-adce-4e1b-b94b-bfd3e81675aa admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:48.737341 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db04758-a911-49b2-8cbc-23d6ba2467f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:48.738145 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db04758-a911-49b2-8cbc-23d6ba2467f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:48.738430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db04758-a911-49b2-8cbc-23d6ba2467f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:48.738819 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db04758-a911-49b2-8cbc-23d6ba2467f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:48.752438 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db04758-a911-49b2-8cbc-23d6ba2467f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:48.758191 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db04758-a911-49b2-8cbc-23d6ba2467f4 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:48.779721 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 981/1930] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:49.002055 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a98b83-6f94-4b40-ae21-9fd923d4248d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:49.003706 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-40a98b83-6f94-4b40-ae21-9fd923d4248d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:49.020137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a98b83-6f94-4b40-ae21-9fd923d4248d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:49.026363 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac7311-a0d1-4e54-99d0-fd28494fb607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:49.028302 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac7311-a0d1-4e54-99d0-fd28494fb607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:49.028733 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac7311-a0d1-4e54-99d0-fd28494fb607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:49.029208 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac7311-a0d1-4e54-99d0-fd28494fb607 admin admin] PATH_INFO: `/v3/projects/e658ba7a93cd4042b45d510a1dfc7005` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:49.030534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac7311-a0d1-4e54-99d0-fd28494fb607 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e658ba7a93cd4042b45d510a1dfc7005)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:49.033424 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac7311-a0d1-4e54-99d0-fd28494fb607 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:49.094250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3bac7311-a0d1-4e54-99d0-fd28494fb607 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e658ba7a93cd4042b45d510a1dfc7005', 'request_id': 'req-3bac7311-a0d1-4e54-99d0-fd28494fb607'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.102567 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3bac7311-a0d1-4e54-99d0-fd28494fb607 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e658ba7a93cd4042b45d510a1dfc7005', 'request_id': 'req-3bac7311-a0d1-4e54-99d0-fd28494fb607'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.109448 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3bac7311-a0d1-4e54-99d0-fd28494fb607 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e658ba7a93cd4042b45d510a1dfc7005', 'request_id': 'req-3bac7311-a0d1-4e54-99d0-fd28494fb607'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.121195 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3bac7311-a0d1-4e54-99d0-fd28494fb607 admin admin] The token cache is being invalidate because project e658ba7a93cd4042b45d510a1dfc7005 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:49.121474 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3bac7311-a0d1-4e54-99d0-fd28494fb607 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.122172 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3bac7311-a0d1-4e54-99d0-fd28494fb607 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.124055 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 950/1931] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:49 2022] DELETE /identity/v3/projects/e658ba7a93cd4042b45d510a1dfc7005 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:49.127876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db04758-a911-49b2-8cbc-23d6ba2467f4 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:49.144113 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db04758-a911-49b2-8cbc-23d6ba2467f4 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:49.204332 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac7311-a0d1-4e54-99d0-fd28494fb607 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:49.207835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce04447-c496-44ef-9a3c-9fb89c769dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:49.209192 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce04447-c496-44ef-9a3c-9fb89c769dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:49.209619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce04447-c496-44ef-9a3c-9fb89c769dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:49.210027 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce04447-c496-44ef-9a3c-9fb89c769dd2 admin admin] PATH_INFO: `/v3/users/24380b69aa9f43d68a44dc810cdd707d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:49.211799 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce04447-c496-44ef-9a3c-9fb89c769dd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24380b69aa9f43d68a44dc810cdd707d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:49.212459 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3bac7311-a0d1-4e54-99d0-fd28494fb607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:49.214798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce04447-c496-44ef-9a3c-9fb89c769dd2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:49.229806 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:49.257000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac7311-a0d1-4e54-99d0-fd28494fb607 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:49.268472 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab65737-7def-4bc6-8a60-b1cc9b2972ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:49.269379 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab65737-7def-4bc6-8a60-b1cc9b2972ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:49.269659 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab65737-7def-4bc6-8a60-b1cc9b2972ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:49.269928 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab65737-7def-4bc6-8a60-b1cc9b2972ae admin admin] PATH_INFO: `/v3/projects/e94dc9bb6911462192ee42f61a1cc784` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:49.275369 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab65737-7def-4bc6-8a60-b1cc9b2972ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=e94dc9bb6911462192ee42f61a1cc784)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:49.277427 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab65737-7def-4bc6-8a60-b1cc9b2972ae admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:49.281983 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2ce04447-c496-44ef-9a3c-9fb89c769dd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24380b69aa9f43d68a44dc810cdd707d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.282646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2ce04447-c496-44ef-9a3c-9fb89c769dd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24380b69aa9f43d68a44dc810cdd707d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.283256 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2ce04447-c496-44ef-9a3c-9fb89c769dd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24380b69aa9f43d68a44dc810cdd707d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.314889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2ce04447-c496-44ef-9a3c-9fb89c769dd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24380b69aa9f43d68a44dc810cdd707d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.324257 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2ce04447-c496-44ef-9a3c-9fb89c769dd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24380b69aa9f43d68a44dc810cdd707d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.346362 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eab65737-7def-4bc6-8a60-b1cc9b2972ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e94dc9bb6911462192ee42f61a1cc784', 'request_id': 'req-eab65737-7def-4bc6-8a60-b1cc9b2972ae'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.346830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 982/1932] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:49 2022] DELETE /identity/v3/users/24380b69aa9f43d68a44dc810cdd707d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:49.371806 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eab65737-7def-4bc6-8a60-b1cc9b2972ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e94dc9bb6911462192ee42f61a1cc784', 'request_id': 'req-eab65737-7def-4bc6-8a60-b1cc9b2972ae'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.385918 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eab65737-7def-4bc6-8a60-b1cc9b2972ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e94dc9bb6911462192ee42f61a1cc784', 'request_id': 'req-eab65737-7def-4bc6-8a60-b1cc9b2972ae'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.402575 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eab65737-7def-4bc6-8a60-b1cc9b2972ae admin admin] The token cache is being invalidate because project e94dc9bb6911462192ee42f61a1cc784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:49.402773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eab65737-7def-4bc6-8a60-b1cc9b2972ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.404053 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eab65737-7def-4bc6-8a60-b1cc9b2972ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.406066 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 951/1933] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:49 2022] DELETE /identity/v3/projects/e94dc9bb6911462192ee42f61a1cc784 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:49.410417 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce04447-c496-44ef-9a3c-9fb89c769dd2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:49.411892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2ce04447-c496-44ef-9a3c-9fb89c769dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:49.492361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce04447-c496-44ef-9a3c-9fb89c769dd2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:49.498543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf50c4f-a138-467a-8a89-9e1aba50638a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:49.499442 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf50c4f-a138-467a-8a89-9e1aba50638a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:49.499763 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf50c4f-a138-467a-8a89-9e1aba50638a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:49.500046 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf50c4f-a138-467a-8a89-9e1aba50638a admin admin] PATH_INFO: `/v3/users/1730eaa01d894396986343ef7b788779` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:49.501572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf50c4f-a138-467a-8a89-9e1aba50638a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1730eaa01d894396986343ef7b788779)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:49.503225 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf50c4f-a138-467a-8a89-9e1aba50638a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:49.565867 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-aaf50c4f-a138-467a-8a89-9e1aba50638a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1730eaa01d894396986343ef7b788779'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.566539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-aaf50c4f-a138-467a-8a89-9e1aba50638a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1730eaa01d894396986343ef7b788779'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.572098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-aaf50c4f-a138-467a-8a89-9e1aba50638a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1730eaa01d894396986343ef7b788779'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.610390 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-aaf50c4f-a138-467a-8a89-9e1aba50638a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1730eaa01d894396986343ef7b788779'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.620365 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-aaf50c4f-a138-467a-8a89-9e1aba50638a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1730eaa01d894396986343ef7b788779'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.637964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 983/1934] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:49 2022] DELETE /identity/v3/users/1730eaa01d894396986343ef7b788779 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:49.733655 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab65737-7def-4bc6-8a60-b1cc9b2972ae admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:49.735195 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-eab65737-7def-4bc6-8a60-b1cc9b2972ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:49.779848 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:49.887006 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab65737-7def-4bc6-8a60-b1cc9b2972ae admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:49.955932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49e872-cd95-4c27-9c47-465fe3acd430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:49.956828 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49e872-cd95-4c27-9c47-465fe3acd430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:49.957116 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49e872-cd95-4c27-9c47-465fe3acd430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:49.957397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49e872-cd95-4c27-9c47-465fe3acd430 admin admin] PATH_INFO: `/v3/projects/b3cc9379a2114b03b195fedd6969120b` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:49.958789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49e872-cd95-4c27-9c47-465fe3acd430 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3cc9379a2114b03b195fedd6969120b)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:49.960881 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49e872-cd95-4c27-9c47-465fe3acd430 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:50.052909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf50c4f-a138-467a-8a89-9e1aba50638a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:50.053637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2f49e872-cd95-4c27-9c47-465fe3acd430 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3cc9379a2114b03b195fedd6969120b', 'request_id': 'req-2f49e872-cd95-4c27-9c47-465fe3acd430'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:50.055706 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-aaf50c4f-a138-467a-8a89-9e1aba50638a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:50.077639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf50c4f-a138-467a-8a89-9e1aba50638a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:50.077792 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2f49e872-cd95-4c27-9c47-465fe3acd430 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3cc9379a2114b03b195fedd6969120b', 'request_id': 'req-2f49e872-cd95-4c27-9c47-465fe3acd430'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:50.085030 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:50.085179 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2f49e872-cd95-4c27-9c47-465fe3acd430 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3cc9379a2114b03b195fedd6969120b', 'request_id': 'req-2f49e872-cd95-4c27-9c47-465fe3acd430'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:50.162293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2f49e872-cd95-4c27-9c47-465fe3acd430 admin admin] The token cache is being invalidate because project b3cc9379a2114b03b195fedd6969120b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:50.163283 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2f49e872-cd95-4c27-9c47-465fe3acd430 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:50.165731 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2f49e872-cd95-4c27-9c47-465fe3acd430 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:50.168290 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 952/1935] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:49 2022] DELETE /identity/v3/projects/b3cc9379a2114b03b195fedd6969120b => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:50.173937 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf71b6d-fe40-4640-b348-18bc5b0394cb None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:50.174475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf71b6d-fe40-4640-b348-18bc5b0394cb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:50.174899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf71b6d-fe40-4640-b348-18bc5b0394cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:50.177792 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e389a-b067-4f6e-8f28-a00fd82ad349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:50.178680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e389a-b067-4f6e-8f28-a00fd82ad349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:50.178958 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e389a-b067-4f6e-8f28-a00fd82ad349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:50.179446 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e389a-b067-4f6e-8f28-a00fd82ad349 admin admin] PATH_INFO: `/v3/projects/406c455d9dfa4804bf14f8a27f1182f3` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:50.180791 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e389a-b067-4f6e-8f28-a00fd82ad349 admin admin] RBAC: Authorizing `identity:delete_project(project_id=406c455d9dfa4804bf14f8a27f1182f3)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:50.182841 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e389a-b067-4f6e-8f28-a00fd82ad349 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:50.212185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-bcf71b6d-fe40-4640-b348-18bc5b0394cb None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:50.213455 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-bcf71b6d-fe40-4640-b348-18bc5b0394cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:50.223634 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 953/1936] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:50.231432 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83864a-69fc-40bf-96a5-df137a63d4b0 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:50.231844 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83864a-69fc-40bf-96a5-df137a63d4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:50.232192 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83864a-69fc-40bf-96a5-df137a63d4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:50.253199 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-510e389a-b067-4f6e-8f28-a00fd82ad349 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '406c455d9dfa4804bf14f8a27f1182f3', 'request_id': 'req-510e389a-b067-4f6e-8f28-a00fd82ad349'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:50.255108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-0e83864a-69fc-40bf-96a5-df137a63d4b0 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:50.256415 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0e83864a-69fc-40bf-96a5-df137a63d4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:50.268697 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-510e389a-b067-4f6e-8f28-a00fd82ad349 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '406c455d9dfa4804bf14f8a27f1182f3', 'request_id': 'req-510e389a-b067-4f6e-8f28-a00fd82ad349'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:50.278123 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-510e389a-b067-4f6e-8f28-a00fd82ad349 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '406c455d9dfa4804bf14f8a27f1182f3', 'request_id': 'req-510e389a-b067-4f6e-8f28-a00fd82ad349'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:50.293414 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-510e389a-b067-4f6e-8f28-a00fd82ad349 admin admin] The token cache is being invalidate because project 406c455d9dfa4804bf14f8a27f1182f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:50.293710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-510e389a-b067-4f6e-8f28-a00fd82ad349 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:50.294377 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-510e389a-b067-4f6e-8f28-a00fd82ad349 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:50.296732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 984/1937] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:50 2022] DELETE /identity/v3/projects/406c455d9dfa4804bf14f8a27f1182f3 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:50.302029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e29fef-cf91-4567-b724-ab4b76437704 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:50.304314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e29fef-cf91-4567-b724-ab4b76437704 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:50.304489 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e29fef-cf91-4567-b724-ab4b76437704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:50.316821 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:50.362046 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-75e29fef-cf91-4567-b724-ab4b76437704 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:50.366874 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-75e29fef-cf91-4567-b724-ab4b76437704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:50.368377 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 954/1938] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:50.379946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83864a-69fc-40bf-96a5-df137a63d4b0 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:50.398314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 985/1939] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:50.406770 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83864a-69fc-40bf-96a5-df137a63d4b0 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:50.408850 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fbb45-279c-4926-b353-01b05b68bc50 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:50.409388 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fbb45-279c-4926-b353-01b05b68bc50 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:50.410110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fbb45-279c-4926-b353-01b05b68bc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:50.412560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1eb73f-a683-4a48-977e-cfb7a799b5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:50.413329 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1eb73f-a683-4a48-977e-cfb7a799b5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:50.413601 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1eb73f-a683-4a48-977e-cfb7a799b5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:50.414012 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1eb73f-a683-4a48-977e-cfb7a799b5a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:50.415102 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1eb73f-a683-4a48-977e-cfb7a799b5a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:50.416120 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1eb73f-a683-4a48-977e-cfb7a799b5a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:50.419279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1eb73f-a683-4a48-977e-cfb7a799b5a4 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:50.438949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 955/1940] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:16:50.444067 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1eb73f-a683-4a48-977e-cfb7a799b5a4 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:50.467116 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1eb73f-a683-4a48-977e-cfb7a799b5a4 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:50.484164 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817285a-f0ad-4f29-a914-962d6938138a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:50.484851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817285a-f0ad-4f29-a914-962d6938138a admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:50.485151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817285a-f0ad-4f29-a914-962d6938138a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:50.485396 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817285a-f0ad-4f29-a914-962d6938138a admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:50.488619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-828fbb45-279c-4926-b353-01b05b68bc50 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:50.489845 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-828fbb45-279c-4926-b353-01b05b68bc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:50.490247 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817285a-f0ad-4f29-a914-962d6938138a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:50.499317 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817285a-f0ad-4f29-a914-962d6938138a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:50.551633 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 986/1941] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:50.551633 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828fbb45-279c-4926-b353-01b05b68bc50 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:50.578687 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828fbb45-279c-4926-b353-01b05b68bc50 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:50.593614 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd01715-b142-40b2-8588-ca64e54d49fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:50.594407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd01715-b142-40b2-8588-ca64e54d49fd admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:50.594768 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd01715-b142-40b2-8588-ca64e54d49fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:50.595049 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd01715-b142-40b2-8588-ca64e54d49fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:50.595960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd01715-b142-40b2-8588-ca64e54d49fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:50.596135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd01715-b142-40b2-8588-ca64e54d49fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:50.598259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd01715-b142-40b2-8588-ca64e54d49fd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:50.621841 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 987/1942] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:16:50.622318 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 956/1943] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:50 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:50.623296 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd01715-b142-40b2-8588-ca64e54d49fd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:50.623454 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817285a-f0ad-4f29-a914-962d6938138a admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:50.650982 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd01715-b142-40b2-8588-ca64e54d49fd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:50.742600 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479cc9a-9b11-44f9-a21b-8d66509fdadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:50.743349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479cc9a-9b11-44f9-a21b-8d66509fdadd admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:50.743669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479cc9a-9b11-44f9-a21b-8d66509fdadd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:50.743951 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479cc9a-9b11-44f9-a21b-8d66509fdadd admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:50.744067 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817285a-f0ad-4f29-a914-962d6938138a admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:50.744729 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479cc9a-9b11-44f9-a21b-8d66509fdadd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:50.746720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479cc9a-9b11-44f9-a21b-8d66509fdadd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:50.750602 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad6376-24dc-4b87-89f5-274854d718b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:50.751576 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad6376-24dc-4b87-89f5-274854d718b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:50.751984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad6376-24dc-4b87-89f5-274854d718b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:50.752355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad6376-24dc-4b87-89f5-274854d718b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:50.753146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad6376-24dc-4b87-89f5-274854d718b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:50.757462 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad6376-24dc-4b87-89f5-274854d718b8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:50.774515 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 988/1944] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:50 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:50.780058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479cc9a-9b11-44f9-a21b-8d66509fdadd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:50.786003 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:50.800645 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479cc9a-9b11-44f9-a21b-8d66509fdadd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:50.818944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 957/1945] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:50 2022] POST /identity/v3/projects => generated 373 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:50.819343 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f106dae-476e-4605-aad1-db9cb3db1ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:50.820076 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f106dae-476e-4605-aad1-db9cb3db1ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:50.820361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f106dae-476e-4605-aad1-db9cb3db1ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:50.820633 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f106dae-476e-4605-aad1-db9cb3db1ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:50.820932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad6376-24dc-4b87-89f5-274854d718b8 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:50.821424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f106dae-476e-4605-aad1-db9cb3db1ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:50.826014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f106dae-476e-4605-aad1-db9cb3db1ca9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:50.851330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 989/1946] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:50.852966 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f106dae-476e-4605-aad1-db9cb3db1ca9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:50.864600 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f106dae-476e-4605-aad1-db9cb3db1ca9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:50.933578 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad6376-24dc-4b87-89f5-274854d718b8 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:50.956454 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c134aad-521e-4567-a49a-1ab2c9198eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:50.959012 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3985b-1c71-4ac0-b4d2-8a4e546ecf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:50.959734 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3985b-1c71-4ac0-b4d2-8a4e546ecf81 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:50.960027 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3985b-1c71-4ac0-b4d2-8a4e546ecf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:50.960678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3985b-1c71-4ac0-b4d2-8a4e546ecf81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:50.961796 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3985b-1c71-4ac0-b4d2-8a4e546ecf81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:50.961994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c134aad-521e-4567-a49a-1ab2c9198eda admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:50.962271 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c134aad-521e-4567-a49a-1ab2c9198eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:50.962557 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c134aad-521e-4567-a49a-1ab2c9198eda admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:50.963314 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c134aad-521e-4567-a49a-1ab2c9198eda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:50.963500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3985b-1c71-4ac0-b4d2-8a4e546ecf81 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:50.965473 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c134aad-521e-4567-a49a-1ab2c9198eda admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:50.990806 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 990/1947] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:16:50.995095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3985b-1c71-4ac0-b4d2-8a4e546ecf81 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:51.021966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 958/1948] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:50 2022] POST /identity/v3/users => generated 287 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:51.025549 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3985b-1c71-4ac0-b4d2-8a4e546ecf81 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:51.026492 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c134aad-521e-4567-a49a-1ab2c9198eda admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:51.031441 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4026dd-52c9-42b6-9131-0f8b8d01959f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:51.033652 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4026dd-52c9-42b6-9131-0f8b8d01959f admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:51.034013 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4026dd-52c9-42b6-9131-0f8b8d01959f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:51.034545 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4026dd-52c9-42b6-9131-0f8b8d01959f admin admin] PATH_INFO: `/v3/projects/910d9eb138824079aa6acd95ab6f0e06/users/547a6e2319d74129b9a08c7baf49e6c9/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:51.056369 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4026dd-52c9-42b6-9131-0f8b8d01959f admin admin] RBAC: Authorizing `identity:create_grant(project_id=910d9eb138824079aa6acd95ab6f0e06, user_id=547a6e2319d74129b9a08c7baf49e6c9, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:51.057552 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c134aad-521e-4567-a49a-1ab2c9198eda admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:51.059455 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4026dd-52c9-42b6-9131-0f8b8d01959f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:51.073042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859201d-9f3e-42dd-9123-f81b36d144fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:51.074377 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859201d-9f3e-42dd-9123-f81b36d144fa admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:51.074664 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859201d-9f3e-42dd-9123-f81b36d144fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:51.074939 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859201d-9f3e-42dd-9123-f81b36d144fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:51.075873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859201d-9f3e-42dd-9123-f81b36d144fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:51.077532 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859201d-9f3e-42dd-9123-f81b36d144fa admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:51.079703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 991/1949] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:50 2022] PUT /identity/v3/projects/910d9eb138824079aa6acd95ab6f0e06/users/547a6e2319d74129b9a08c7baf49e6c9/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:51.093788 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aad58b-6ccf-4abe-a895-bdf266d62e1e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:51.094645 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aad58b-6ccf-4abe-a895-bdf266d62e1e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:51.095004 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aad58b-6ccf-4abe-a895-bdf266d62e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:51.108776 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 959/1950] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:51.113109 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859201d-9f3e-42dd-9123-f81b36d144fa admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:51.127080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859201d-9f3e-42dd-9123-f81b36d144fa admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:51.137085 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b5aad58b-6ccf-4abe-a895-bdf266d62e1e None None] MFA Rules not processed for user `547a6e2319d74129b9a08c7baf49e6c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:51.138818 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b5aad58b-6ccf-4abe-a895-bdf266d62e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:51.146788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:51.243561 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 992/1951] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:51.249889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aad58b-6ccf-4abe-a895-bdf266d62e1e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:51.270790 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d43bd-c249-4086-8140-cdfd9b63a413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:51.272540 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d43bd-c249-4086-8140-cdfd9b63a413 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:51.272828 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d43bd-c249-4086-8140-cdfd9b63a413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:51.273111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d43bd-c249-4086-8140-cdfd9b63a413 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:51.274893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d43bd-c249-4086-8140-cdfd9b63a413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:51.291925 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d43bd-c249-4086-8140-cdfd9b63a413 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:51.305165 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 960/1952] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:51.308742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d43bd-c249-4086-8140-cdfd9b63a413 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:51.320892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d43bd-c249-4086-8140-cdfd9b63a413 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:51.326849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15623157-ecc6-434b-81d1-e4a6af6b81f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:51.328090 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15623157-ecc6-434b-81d1-e4a6af6b81f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:51.328529 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15623157-ecc6-434b-81d1-e4a6af6b81f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:51.328919 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15623157-ecc6-434b-81d1-e4a6af6b81f6 admin admin] PATH_INFO: `/v3/projects/52d2c8634d374385a59c1ffe45c0c95e/users/5d6cd7874448444f9ff4a7637b161e0f/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:51.346522 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15623157-ecc6-434b-81d1-e4a6af6b81f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d2c8634d374385a59c1ffe45c0c95e, user_id=5d6cd7874448444f9ff4a7637b161e0f, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:51.357913 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15623157-ecc6-434b-81d1-e4a6af6b81f6 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:51.366387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:51.368437 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 961/1953] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:51 2022] PUT /identity/v3/projects/52d2c8634d374385a59c1ffe45c0c95e/users/5d6cd7874448444f9ff4a7637b161e0f/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:51.379301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f06e7-4d94-4b35-ae76-f462f94daa27 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:51.379888 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f06e7-4d94-4b35-ae76-f462f94daa27 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:51.380254 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f06e7-4d94-4b35-ae76-f462f94daa27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:51.387675 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aad58b-6ccf-4abe-a895-bdf266d62e1e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:51.433853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b38f06e7-4d94-4b35-ae76-f462f94daa27 None None] MFA Rules not processed for user `5d6cd7874448444f9ff4a7637b161e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:51.436732 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b38f06e7-4d94-4b35-ae76-f462f94daa27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:51.480246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241efc9d-01d3-4af6-b8da-179d335e8282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:51.481159 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241efc9d-01d3-4af6-b8da-179d335e8282 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:51.481465 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241efc9d-01d3-4af6-b8da-179d335e8282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:51.481740 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241efc9d-01d3-4af6-b8da-179d335e8282 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:51.482457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241efc9d-01d3-4af6-b8da-179d335e8282 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:51.484399 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241efc9d-01d3-4af6-b8da-179d335e8282 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:51.505446 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 962/1954] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:51.509932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f06e7-4d94-4b35-ae76-f462f94daa27 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:51.513345 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 993/1955] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:51 2022] POST /identity/v3/projects => generated 383 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:51.514653 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241efc9d-01d3-4af6-b8da-179d335e8282 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:51.521131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f06e7-4d94-4b35-ae76-f462f94daa27 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:51.577792 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241efc9d-01d3-4af6-b8da-179d335e8282 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:51.580719 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9943b-efe3-4c88-b23e-6a2ce82dd018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:51.581398 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9943b-efe3-4c88-b23e-6a2ce82dd018 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:51.581680 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9943b-efe3-4c88-b23e-6a2ce82dd018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:51.581958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9943b-efe3-4c88-b23e-6a2ce82dd018 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:51.582436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98cf0b-d1e8-4af8-92fb-ae2e0680263e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:51.582655 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9943b-efe3-4c88-b23e-6a2ce82dd018 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:51.583110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98cf0b-d1e8-4af8-92fb-ae2e0680263e admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:51.583389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98cf0b-d1e8-4af8-92fb-ae2e0680263e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:51.583689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98cf0b-d1e8-4af8-92fb-ae2e0680263e admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:51.584379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98cf0b-d1e8-4af8-92fb-ae2e0680263e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:51.584636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9943b-efe3-4c88-b23e-6a2ce82dd018 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:51.586291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98cf0b-d1e8-4af8-92fb-ae2e0680263e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:51.609100 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 994/1956] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:51 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:51.610051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:51.612831 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98cf0b-d1e8-4af8-92fb-ae2e0680263e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:51.624278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98cf0b-d1e8-4af8-92fb-ae2e0680263e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:51.629056 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1e06a-5daf-40c5-b3bb-a2beca477dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:51.629729 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1e06a-5daf-40c5-b3bb-a2beca477dde admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:51.630058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1e06a-5daf-40c5-b3bb-a2beca477dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:51.630338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1e06a-5daf-40c5-b3bb-a2beca477dde admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:51.631063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f1e06a-5daf-40c5-b3bb-a2beca477dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:51.632712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f1e06a-5daf-40c5-b3bb-a2beca477dde admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:51.634515 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 963/1957] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:51 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:51.638571 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9943b-efe3-4c88-b23e-6a2ce82dd018 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:51.648839 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 995/1958] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:51.652399 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1e06a-5daf-40c5-b3bb-a2beca477dde admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:51.663124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1e06a-5daf-40c5-b3bb-a2beca477dde admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:51.758272 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb50a60-af83-4797-9727-041777dd4def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:51.760936 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb50a60-af83-4797-9727-041777dd4def admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:51.761352 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb50a60-af83-4797-9727-041777dd4def admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:51.761810 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb50a60-af83-4797-9727-041777dd4def admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:51.762572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb50a60-af83-4797-9727-041777dd4def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:51.764828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb50a60-af83-4797-9727-041777dd4def admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:51.766077 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9943b-efe3-4c88-b23e-6a2ce82dd018 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:51.776009 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:51.785900 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e2471-8ff7-453d-96df-7690d581b39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:51.786776 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e2471-8ff7-453d-96df-7690d581b39c admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:51.787107 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e2471-8ff7-453d-96df-7690d581b39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:51.787359 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e2471-8ff7-453d-96df-7690d581b39c admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:51.788433 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e2471-8ff7-453d-96df-7690d581b39c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:51.792701 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e2471-8ff7-453d-96df-7690d581b39c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:51.797248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 996/1959] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:51.798911 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb50a60-af83-4797-9727-041777dd4def admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:51.815325 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb50a60-af83-4797-9727-041777dd4def admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:51.837842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 964/1960] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:51 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:51.840313 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233eb7e-9832-406e-8ae9-179eb0f41e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:51.841347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233eb7e-9832-406e-8ae9-179eb0f41e8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:51.841633 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233eb7e-9832-406e-8ae9-179eb0f41e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:51.841921 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233eb7e-9832-406e-8ae9-179eb0f41e8e admin admin] PATH_INFO: `/v3/projects/66476f9b51e54d6c8e2f68c30b09c0bc/users/a1f01fc03b6449faa41c76aa03ed281e/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:51.842706 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e2471-8ff7-453d-96df-7690d581b39c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:51.860266 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e2471-8ff7-453d-96df-7690d581b39c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:51.867644 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69002f0-8b5f-4e4c-b545-a794e9196ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:51.868289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69002f0-8b5f-4e4c-b545-a794e9196ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:51.868422 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69002f0-8b5f-4e4c-b545-a794e9196ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:51.868682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69002f0-8b5f-4e4c-b545-a794e9196ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:51.869914 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69002f0-8b5f-4e4c-b545-a794e9196ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:51.873437 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69002f0-8b5f-4e4c-b545-a794e9196ee9 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:51.874558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233eb7e-9832-406e-8ae9-179eb0f41e8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=66476f9b51e54d6c8e2f68c30b09c0bc, user_id=a1f01fc03b6449faa41c76aa03ed281e, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:51.880059 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233eb7e-9832-406e-8ae9-179eb0f41e8e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:51.890233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 965/1961] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:16:51.896728 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69002f0-8b5f-4e4c-b545-a794e9196ee9 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:51.912971 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69002f0-8b5f-4e4c-b545-a794e9196ee9 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:51.927705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 997/1962] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:51 2022] PUT /identity/v3/projects/66476f9b51e54d6c8e2f68c30b09c0bc/users/a1f01fc03b6449faa41c76aa03ed281e/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:51.929529 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8b755-385b-42d9-954d-d74b7b4b2f75 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:51.929894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8b755-385b-42d9-954d-d74b7b4b2f75 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:51.930179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8b755-385b-42d9-954d-d74b7b4b2f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:51.983682 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-23d8b755-385b-42d9-954d-d74b7b4b2f75 None None] MFA Rules not processed for user `a1f01fc03b6449faa41c76aa03ed281e`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:51.985108 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-23d8b755-385b-42d9-954d-d74b7b4b2f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:52.004664 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e26fd1-b851-47d1-ad12-69ce031407a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:52.005583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e26fd1-b851-47d1-ad12-69ce031407a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:52.005864 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e26fd1-b851-47d1-ad12-69ce031407a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:52.006139 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e26fd1-b851-47d1-ad12-69ce031407a8 admin admin] PATH_INFO: `/v3/projects/e8363f2d11654c95b32386104a420b16/users/b41534f825ae4ca7851cd2e703b74bac/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:52.028062 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e26fd1-b851-47d1-ad12-69ce031407a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8363f2d11654c95b32386104a420b16, user_id=b41534f825ae4ca7851cd2e703b74bac, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:52.031420 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e26fd1-b851-47d1-ad12-69ce031407a8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:52.055045 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 966/1963] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:51 2022] PUT /identity/v3/projects/e8363f2d11654c95b32386104a420b16/users/b41534f825ae4ca7851cd2e703b74bac/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:52.059113 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e26fd1-b851-47d1-ad12-69ce031407a8 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:52.071550 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 998/1964] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:52.075840 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8b755-385b-42d9-954d-d74b7b4b2f75 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:52.087147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8b755-385b-42d9-954d-d74b7b4b2f75 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:52.134281 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e26fd1-b851-47d1-ad12-69ce031407a8 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:52.141127 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604148aa-acf0-4b6e-9e61-739292861d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:52.143105 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604148aa-acf0-4b6e-9e61-739292861d74 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:52.143578 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604148aa-acf0-4b6e-9e61-739292861d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:52.144656 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604148aa-acf0-4b6e-9e61-739292861d74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:52.145634 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604148aa-acf0-4b6e-9e61-739292861d74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:52.147762 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604148aa-acf0-4b6e-9e61-739292861d74 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:52.159451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:52.165872 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 967/1965] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:16:52.173041 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604148aa-acf0-4b6e-9e61-739292861d74 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:52.178396 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2dc320-472e-44f2-9bb2-49d29e297951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:52.179777 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2dc320-472e-44f2-9bb2-49d29e297951 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:52.180119 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2dc320-472e-44f2-9bb2-49d29e297951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:52.180464 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2dc320-472e-44f2-9bb2-49d29e297951 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:52.182788 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2dc320-472e-44f2-9bb2-49d29e297951 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:52.185692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2dc320-472e-44f2-9bb2-49d29e297951 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:52.186227 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604148aa-acf0-4b6e-9e61-739292861d74 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:52.193884 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e7959-8608-46d5-8a35-b14d6a2fd497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:52.194917 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e7959-8608-46d5-8a35-b14d6a2fd497 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:52.195094 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e7959-8608-46d5-8a35-b14d6a2fd497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:52.195585 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e7959-8608-46d5-8a35-b14d6a2fd497 admin admin] PATH_INFO: `/v3/projects/e8363f2d11654c95b32386104a420b16/users/b41534f825ae4ca7851cd2e703b74bac/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:52.198214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e7959-8608-46d5-8a35-b14d6a2fd497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8363f2d11654c95b32386104a420b16, user_id=b41534f825ae4ca7851cd2e703b74bac, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:52.201476 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e7959-8608-46d5-8a35-b14d6a2fd497 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:52.215525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 968/1966] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:52 2022] PUT /identity/v3/projects/e8363f2d11654c95b32386104a420b16/users/b41534f825ae4ca7851cd2e703b74bac/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:52.225808 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fe044-536a-434b-a4b9-3253d51db248 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:52.228614 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fe044-536a-434b-a4b9-3253d51db248 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:52.228614 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fe044-536a-434b-a4b9-3253d51db248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:52.237062 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 999/1967] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:52 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:52.241377 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2dc320-472e-44f2-9bb2-49d29e297951 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:52.262653 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-412fe044-536a-434b-a4b9-3253d51db248 None None] MFA Rules not processed for user `b41534f825ae4ca7851cd2e703b74bac`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:52.264599 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-412fe044-536a-434b-a4b9-3253d51db248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:52.302828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:52.315680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2dc320-472e-44f2-9bb2-49d29e297951 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:52.320280 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb659a7-7096-420a-9d88-9bacf291f578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:52.320957 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb659a7-7096-420a-9d88-9bacf291f578 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:52.321281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb659a7-7096-420a-9d88-9bacf291f578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:52.321517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb659a7-7096-420a-9d88-9bacf291f578 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:52.322242 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb659a7-7096-420a-9d88-9bacf291f578 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:52.324230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb659a7-7096-420a-9d88-9bacf291f578 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:52.336553 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 969/1968] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:52.346616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1000/1969] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:52 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:52.347778 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fe044-536a-434b-a4b9-3253d51db248 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:52.347900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb659a7-7096-420a-9d88-9bacf291f578 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:52.349378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fbb659a7-7096-420a-9d88-9bacf291f578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:52.358474 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fe044-536a-434b-a4b9-3253d51db248 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:52.363781 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b083d-41fa-4307-8bd7-cc2acc61ee60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:52.364448 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b083d-41fa-4307-8bd7-cc2acc61ee60 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:52.364975 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b083d-41fa-4307-8bd7-cc2acc61ee60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:52.364975 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b083d-41fa-4307-8bd7-cc2acc61ee60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:52.365717 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b083d-41fa-4307-8bd7-cc2acc61ee60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:52.367317 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b083d-41fa-4307-8bd7-cc2acc61ee60 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:52.380305 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 970/1970] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:52.384560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b083d-41fa-4307-8bd7-cc2acc61ee60 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:52.396236 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b083d-41fa-4307-8bd7-cc2acc61ee60 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:52.401195 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fac91e-0fe6-431a-b266-2b5fefc3bab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:52.402097 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fac91e-0fe6-431a-b266-2b5fefc3bab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:52.402388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fac91e-0fe6-431a-b266-2b5fefc3bab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:52.402661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fac91e-0fe6-431a-b266-2b5fefc3bab9 admin admin] PATH_INFO: `/v3/projects/eacde9cac83647bfbb71197d4bf3ba79/users/59277683010b460f972ccb3592314277/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:52.419131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fac91e-0fe6-431a-b266-2b5fefc3bab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eacde9cac83647bfbb71197d4bf3ba79, user_id=59277683010b460f972ccb3592314277, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:52.419947 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb659a7-7096-420a-9d88-9bacf291f578 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:52.422054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fac91e-0fe6-431a-b266-2b5fefc3bab9 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:52.423949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5321b4d-3f5d-40e3-8c6d-56f9894e2002 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:52.424784 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5321b4d-3f5d-40e3-8c6d-56f9894e2002 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:52.425011 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5321b4d-3f5d-40e3-8c6d-56f9894e2002 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:52.425422 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5321b4d-3f5d-40e3-8c6d-56f9894e2002 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:52.436832 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 971/1971] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:52 2022] PUT /identity/v3/projects/eacde9cac83647bfbb71197d4bf3ba79/users/59277683010b460f972ccb3592314277/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:52.440855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fac91e-0fe6-431a-b266-2b5fefc3bab9 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:52.521532 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5321b4d-3f5d-40e3-8c6d-56f9894e2002 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:52.524505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5321b4d-3f5d-40e3-8c6d-56f9894e2002 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:52.524656 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fac91e-0fe6-431a-b266-2b5fefc3bab9 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:52.530206 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5dc2c2-12a7-425a-b45a-7a4af2455ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:52.530898 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5dc2c2-12a7-425a-b45a-7a4af2455ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:52.531206 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5dc2c2-12a7-425a-b45a-7a4af2455ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:52.531636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5dc2c2-12a7-425a-b45a-7a4af2455ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:52.532382 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5dc2c2-12a7-425a-b45a-7a4af2455ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:52.534055 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5dc2c2-12a7-425a-b45a-7a4af2455ab3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:52.535036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1001/1972] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:52.547652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 972/1973] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:16:52.549889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5321b4d-3f5d-40e3-8c6d-56f9894e2002 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:52.563627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5321b4d-3f5d-40e3-8c6d-56f9894e2002 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:52.568551 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de7bd4-845e-48eb-b2c3-41c7e1a47322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:52.569456 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de7bd4-845e-48eb-b2c3-41c7e1a47322 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:52.569742 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de7bd4-845e-48eb-b2c3-41c7e1a47322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:52.570031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de7bd4-845e-48eb-b2c3-41c7e1a47322 admin admin] PATH_INFO: `/v3/projects/eacde9cac83647bfbb71197d4bf3ba79/users/59277683010b460f972ccb3592314277/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:52.572081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de7bd4-845e-48eb-b2c3-41c7e1a47322 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eacde9cac83647bfbb71197d4bf3ba79, user_id=59277683010b460f972ccb3592314277, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:52.574740 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de7bd4-845e-48eb-b2c3-41c7e1a47322 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:52.586856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1002/1974] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:52 2022] PUT /identity/v3/projects/eacde9cac83647bfbb71197d4bf3ba79/users/59277683010b460f972ccb3592314277/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:52.597364 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1317ab6-1f03-4d6e-8dce-a165065a3778 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:52.597649 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1317ab6-1f03-4d6e-8dce-a165065a3778 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:52.597929 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1317ab6-1f03-4d6e-8dce-a165065a3778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:52.625476 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e1317ab6-1f03-4d6e-8dce-a165065a3778 None None] MFA Rules not processed for user `59277683010b460f972ccb3592314277`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:52.626716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e1317ab6-1f03-4d6e-8dce-a165065a3778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:52.645368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:52.706103 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 973/1975] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:52 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:52.718411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de7bd4-845e-48eb-b2c3-41c7e1a47322 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:52.720242 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a9de7bd4-845e-48eb-b2c3-41c7e1a47322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:52.787374 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de7bd4-845e-48eb-b2c3-41c7e1a47322 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:52.792829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336eeb1d-1d8f-4c5b-89dc-3270783e8c18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:52.793615 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336eeb1d-1d8f-4c5b-89dc-3270783e8c18 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:52.794070 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336eeb1d-1d8f-4c5b-89dc-3270783e8c18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:52.794216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336eeb1d-1d8f-4c5b-89dc-3270783e8c18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:52.854239 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:52.889238 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1317ab6-1f03-4d6e-8dce-a165065a3778 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:52.889382 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336eeb1d-1d8f-4c5b-89dc-3270783e8c18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:52.890872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336eeb1d-1d8f-4c5b-89dc-3270783e8c18 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:52.890979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e1317ab6-1f03-4d6e-8dce-a165065a3778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:52.902844 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1003/1976] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:16:52.970093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1317ab6-1f03-4d6e-8dce-a165065a3778 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:52.984440 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7a4ea-4fba-42d0-a902-b735cc40f10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:52.985618 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7a4ea-4fba-42d0-a902-b735cc40f10f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:52.986094 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7a4ea-4fba-42d0-a902-b735cc40f10f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:52.986363 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7a4ea-4fba-42d0-a902-b735cc40f10f admin admin] PATH_INFO: `/v3/users/accef9f6fe3b4f7a9459a846998e0594` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:52.987705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7a4ea-4fba-42d0-a902-b735cc40f10f admin admin] RBAC: Authorizing `identity:delete_user(user_id=accef9f6fe3b4f7a9459a846998e0594)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:52.990430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7a4ea-4fba-42d0-a902-b735cc40f10f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:53.054393 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f7e7a4ea-4fba-42d0-a902-b735cc40f10f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'accef9f6fe3b4f7a9459a846998e0594'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:53.055358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f7e7a4ea-4fba-42d0-a902-b735cc40f10f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'accef9f6fe3b4f7a9459a846998e0594'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:53.056170 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f7e7a4ea-4fba-42d0-a902-b735cc40f10f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'accef9f6fe3b4f7a9459a846998e0594'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:53.075371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f7e7a4ea-4fba-42d0-a902-b735cc40f10f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'accef9f6fe3b4f7a9459a846998e0594'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:53.084289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f7e7a4ea-4fba-42d0-a902-b735cc40f10f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'accef9f6fe3b4f7a9459a846998e0594'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:53.100317 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 974/1977] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:52 2022] DELETE /identity/v3/users/accef9f6fe3b4f7a9459a846998e0594 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:53.104687 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fac573-8e35-4aef-a6ff-cb49c4eb6b8a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:53.104983 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fac573-8e35-4aef-a6ff-cb49c4eb6b8a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:53.105287 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fac573-8e35-4aef-a6ff-cb49c4eb6b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:53.124653 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a8fac573-8e35-4aef-a6ff-cb49c4eb6b8a None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:53.126465 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a8fac573-8e35-4aef-a6ff-cb49c4eb6b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:53.258514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1004/1978] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:53.270700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7a4ea-4fba-42d0-a902-b735cc40f10f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:53.272808 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f7e7a4ea-4fba-42d0-a902-b735cc40f10f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:53.376087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:53.400237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7a4ea-4fba-42d0-a902-b735cc40f10f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:53.404190 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dcc6d5-2a85-4da8-b445-8a7241408d47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:53.405033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcc6d5-2a85-4da8-b445-8a7241408d47 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:53.405314 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcc6d5-2a85-4da8-b445-8a7241408d47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:53.405587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcc6d5-2a85-4da8-b445-8a7241408d47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:53.415712 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dcc6d5-2a85-4da8-b445-8a7241408d47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:53.417248 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dcc6d5-2a85-4da8-b445-8a7241408d47 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:53.428631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 975/1979] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:53.788279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fac573-8e35-4aef-a6ff-cb49c4eb6b8a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:53.790220 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a8fac573-8e35-4aef-a6ff-cb49c4eb6b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:53.803072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fac573-8e35-4aef-a6ff-cb49c4eb6b8a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:53.808294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e433aaa-cc9b-432a-a89c-b5451718d6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:53.809174 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e433aaa-cc9b-432a-a89c-b5451718d6e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:53.809447 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e433aaa-cc9b-432a-a89c-b5451718d6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:53.809720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e433aaa-cc9b-432a-a89c-b5451718d6e7 admin admin] PATH_INFO: `/v3/projects/32b7a2210a464addbb01905f27b1db48` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:53.811358 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e433aaa-cc9b-432a-a89c-b5451718d6e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32b7a2210a464addbb01905f27b1db48)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:53.812990 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e433aaa-cc9b-432a-a89c-b5451718d6e7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:53.875151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3e433aaa-cc9b-432a-a89c-b5451718d6e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32b7a2210a464addbb01905f27b1db48', 'request_id': 'req-3e433aaa-cc9b-432a-a89c-b5451718d6e7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:53.885374 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3e433aaa-cc9b-432a-a89c-b5451718d6e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32b7a2210a464addbb01905f27b1db48', 'request_id': 'req-3e433aaa-cc9b-432a-a89c-b5451718d6e7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:53.899088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3e433aaa-cc9b-432a-a89c-b5451718d6e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32b7a2210a464addbb01905f27b1db48', 'request_id': 'req-3e433aaa-cc9b-432a-a89c-b5451718d6e7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:53.906800 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3e433aaa-cc9b-432a-a89c-b5451718d6e7 admin admin] The token cache is being invalidate because project 32b7a2210a464addbb01905f27b1db48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:53.907096 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3e433aaa-cc9b-432a-a89c-b5451718d6e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:53.907809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3e433aaa-cc9b-432a-a89c-b5451718d6e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:53.909693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1005/1980] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:53 2022] DELETE /identity/v3/projects/32b7a2210a464addbb01905f27b1db48 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:53.913766 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dcc6d5-2a85-4da8-b445-8a7241408d47 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:53.924250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dcc6d5-2a85-4da8-b445-8a7241408d47 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:53.986213 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86ce50-5307-4384-a9fa-c6343f1ffb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:53.987112 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86ce50-5307-4384-a9fa-c6343f1ffb0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:53.987398 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86ce50-5307-4384-a9fa-c6343f1ffb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:53.987716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86ce50-5307-4384-a9fa-c6343f1ffb0b admin admin] PATH_INFO: `/v3/users/4b5007f27ea843dd821b22e99e737d18` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:53.988998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86ce50-5307-4384-a9fa-c6343f1ffb0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b5007f27ea843dd821b22e99e737d18)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:53.990950 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86ce50-5307-4384-a9fa-c6343f1ffb0b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:54.059913 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fc86ce50-5307-4384-a9fa-c6343f1ffb0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b5007f27ea843dd821b22e99e737d18'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:54.060648 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fc86ce50-5307-4384-a9fa-c6343f1ffb0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b5007f27ea843dd821b22e99e737d18'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:54.061327 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fc86ce50-5307-4384-a9fa-c6343f1ffb0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b5007f27ea843dd821b22e99e737d18'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:54.079022 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fc86ce50-5307-4384-a9fa-c6343f1ffb0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b5007f27ea843dd821b22e99e737d18'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:54.086424 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fc86ce50-5307-4384-a9fa-c6343f1ffb0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b5007f27ea843dd821b22e99e737d18'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:54.101600 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 976/1981] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:53 2022] DELETE /identity/v3/users/4b5007f27ea843dd821b22e99e737d18 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:54.457571 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e433aaa-cc9b-432a-a89c-b5451718d6e7 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:54.459101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3e433aaa-cc9b-432a-a89c-b5451718d6e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:54.480638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e433aaa-cc9b-432a-a89c-b5451718d6e7 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:54.534959 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86ce50-5307-4384-a9fa-c6343f1ffb0b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:54.540306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fc86ce50-5307-4384-a9fa-c6343f1ffb0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:54.557842 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280504d5-e04d-4dd5-82d0-1c25e29d5746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:54.564077 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280504d5-e04d-4dd5-82d0-1c25e29d5746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:54.564077 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280504d5-e04d-4dd5-82d0-1c25e29d5746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:54.564077 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280504d5-e04d-4dd5-82d0-1c25e29d5746 admin admin] PATH_INFO: `/v3/projects/9205bb90c58d48d588f164eb67ce3a5b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:54.564077 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280504d5-e04d-4dd5-82d0-1c25e29d5746 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9205bb90c58d48d588f164eb67ce3a5b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:54.564077 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280504d5-e04d-4dd5-82d0-1c25e29d5746 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:54.566427 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:54.623818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86ce50-5307-4384-a9fa-c6343f1ffb0b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:54.629080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c9c14-59f2-4244-bdcb-ac1dac00a7cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:54.630207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c9c14-59f2-4244-bdcb-ac1dac00a7cc service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:54.630727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c9c14-59f2-4244-bdcb-ac1dac00a7cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:54.631207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c9c14-59f2-4244-bdcb-ac1dac00a7cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:54.633335 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-156c9c14-59f2-4244-bdcb-ac1dac00a7cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:54.644899 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-280504d5-e04d-4dd5-82d0-1c25e29d5746 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9205bb90c58d48d588f164eb67ce3a5b', 'request_id': 'req-280504d5-e04d-4dd5-82d0-1c25e29d5746'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:54.647260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:54.654576 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-280504d5-e04d-4dd5-82d0-1c25e29d5746 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9205bb90c58d48d588f164eb67ce3a5b', 'request_id': 'req-280504d5-e04d-4dd5-82d0-1c25e29d5746'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:54.664393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-280504d5-e04d-4dd5-82d0-1c25e29d5746 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9205bb90c58d48d588f164eb67ce3a5b', 'request_id': 'req-280504d5-e04d-4dd5-82d0-1c25e29d5746'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:54.676943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-280504d5-e04d-4dd5-82d0-1c25e29d5746 admin admin] The token cache is being invalidate because project 9205bb90c58d48d588f164eb67ce3a5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:54.677281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-280504d5-e04d-4dd5-82d0-1c25e29d5746 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:54.677994 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-280504d5-e04d-4dd5-82d0-1c25e29d5746 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:54.680569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1006/1982] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:54 2022] DELETE /identity/v3/projects/9205bb90c58d48d588f164eb67ce3a5b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:54.685699 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65e82e-1a97-45b4-875e-05b5bafe50b8 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:54.686549 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65e82e-1a97-45b4-875e-05b5bafe50b8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:54.686549 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65e82e-1a97-45b4-875e-05b5bafe50b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:54.710632 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0d65e82e-1a97-45b4-875e-05b5bafe50b8 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:54.711997 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0d65e82e-1a97-45b4-875e-05b5bafe50b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:54.712261 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c9c14-59f2-4244-bdcb-ac1dac00a7cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:54.713766 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c9c14-59f2-4244-bdcb-ac1dac00a7cc service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:54.724779 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 977/1983] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:16:54.779764 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1007/1984] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:54.785097 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab144c-279b-43c2-afe9-e757e400a171 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:54.785383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab144c-279b-43c2-afe9-e757e400a171 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:54.785659 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab144c-279b-43c2-afe9-e757e400a171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:54.806727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d3ab144c-279b-43c2-afe9-e757e400a171 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:54.807995 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d3ab144c-279b-43c2-afe9-e757e400a171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:54.818269 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 978/1985] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:54.822376 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65e82e-1a97-45b4-875e-05b5bafe50b8 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:54.833738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65e82e-1a97-45b4-875e-05b5bafe50b8 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:54.838616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83574bb-5c84-4d50-8108-a172c0c9e86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:54.839241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83574bb-5c84-4d50-8108-a172c0c9e86b admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:54.839539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83574bb-5c84-4d50-8108-a172c0c9e86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:54.840185 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83574bb-5c84-4d50-8108-a172c0c9e86b admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:54.840545 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83574bb-5c84-4d50-8108-a172c0c9e86b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:54.840852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83574bb-5c84-4d50-8108-a172c0c9e86b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:54.842499 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83574bb-5c84-4d50-8108-a172c0c9e86b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:54.855672 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1008/1986] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:16:54.860205 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab144c-279b-43c2-afe9-e757e400a171 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:54.871228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab144c-279b-43c2-afe9-e757e400a171 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:54.878930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96340d44-5d84-441b-94cb-c8ef7f461448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:54.879660 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96340d44-5d84-441b-94cb-c8ef7f461448 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:54.879935 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96340d44-5d84-441b-94cb-c8ef7f461448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:54.880220 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96340d44-5d84-441b-94cb-c8ef7f461448 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:54.881001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96340d44-5d84-441b-94cb-c8ef7f461448 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:54.882961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96340d44-5d84-441b-94cb-c8ef7f461448 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:54.914926 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 979/1987] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:54 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:54.919210 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83574bb-5c84-4d50-8108-a172c0c9e86b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:54.929523 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83574bb-5c84-4d50-8108-a172c0c9e86b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:55.001611 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08c99b-e46f-4a97-9601-39de818b78fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:55.002304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08c99b-e46f-4a97-9601-39de818b78fa admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.002592 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08c99b-e46f-4a97-9601-39de818b78fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.002985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08c99b-e46f-4a97-9601-39de818b78fa admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.003801 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb08c99b-e46f-4a97-9601-39de818b78fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:55.006273 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb08c99b-e46f-4a97-9601-39de818b78fa admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:55.031539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1009/1988] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:54 2022] POST /identity/v3/users => generated 285 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:55.035408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96340d44-5d84-441b-94cb-c8ef7f461448 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:55.048256 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96340d44-5d84-441b-94cb-c8ef7f461448 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:55.053734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376d3f0-565d-4b90-b50c-93916bcb9d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:55.054452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376d3f0-565d-4b90-b50c-93916bcb9d9c admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.054739 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376d3f0-565d-4b90-b50c-93916bcb9d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.055036 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376d3f0-565d-4b90-b50c-93916bcb9d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.055788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376d3f0-565d-4b90-b50c-93916bcb9d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:55.057407 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376d3f0-565d-4b90-b50c-93916bcb9d9c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:55.068121 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 980/1989] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:16:55.071880 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08c99b-e46f-4a97-9601-39de818b78fa admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:55.082065 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08c99b-e46f-4a97-9601-39de818b78fa admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:55.086678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6ae82-5a87-4879-859b-5fc74ed2162c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:55.087357 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6ae82-5a87-4879-859b-5fc74ed2162c admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.087670 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6ae82-5a87-4879-859b-5fc74ed2162c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.087951 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6ae82-5a87-4879-859b-5fc74ed2162c admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.088679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6ae82-5a87-4879-859b-5fc74ed2162c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:55.090296 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6ae82-5a87-4879-859b-5fc74ed2162c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:55.091786 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:55.103660 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1010/1990] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:55.108346 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376d3f0-565d-4b90-b50c-93916bcb9d9c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:55.118546 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376d3f0-565d-4b90-b50c-93916bcb9d9c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:55.123248 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8b969-5d74-4b18-a75b-1ee2baf3b2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:55.124184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8b969-5d74-4b18-a75b-1ee2baf3b2f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.124471 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8b969-5d74-4b18-a75b-1ee2baf3b2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.124750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8b969-5d74-4b18-a75b-1ee2baf3b2f8 admin admin] PATH_INFO: `/v3/projects/ae1696d8e86649b88a1aefb4e8614f8b/users/9234efa022e54c828195abea4dd67acf/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.140535 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8b969-5d74-4b18-a75b-1ee2baf3b2f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae1696d8e86649b88a1aefb4e8614f8b, user_id=9234efa022e54c828195abea4dd67acf, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:55.143699 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8b969-5d74-4b18-a75b-1ee2baf3b2f8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:55.159105 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 981/1991] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:55 2022] PUT /identity/v3/projects/ae1696d8e86649b88a1aefb4e8614f8b/users/9234efa022e54c828195abea4dd67acf/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:55.169710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb294e6-c8fd-4186-9eb7-ceef5c6a67e1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.170002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb294e6-c8fd-4186-9eb7-ceef5c6a67e1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.170280 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb294e6-c8fd-4186-9eb7-ceef5c6a67e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.200531 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7eb294e6-c8fd-4186-9eb7-ceef5c6a67e1 None None] MFA Rules not processed for user `9234efa022e54c828195abea4dd67acf`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:55.201753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7eb294e6-c8fd-4186-9eb7-ceef5c6a67e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:55.286668 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1011/1992] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:55.291046 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8b969-5d74-4b18-a75b-1ee2baf3b2f8 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:55.401708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8b969-5d74-4b18-a75b-1ee2baf3b2f8 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:55.410780 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c172013-5685-4ec9-aeaa-a44a0ad17cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:55.412043 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c172013-5685-4ec9-aeaa-a44a0ad17cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.412508 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c172013-5685-4ec9-aeaa-a44a0ad17cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.412929 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c172013-5685-4ec9-aeaa-a44a0ad17cd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.413840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c172013-5685-4ec9-aeaa-a44a0ad17cd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:55.416218 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c172013-5685-4ec9-aeaa-a44a0ad17cd4 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:55.459892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb294e6-c8fd-4186-9eb7-ceef5c6a67e1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:55.461747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7eb294e6-c8fd-4186-9eb7-ceef5c6a67e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:55.463124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 982/1993] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:55 2022] POST /identity/v3/projects => generated 367 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:55.468639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c172013-5685-4ec9-aeaa-a44a0ad17cd4 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:55.486128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb294e6-c8fd-4186-9eb7-ceef5c6a67e1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:55.520506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:55.581309 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee15588-c1df-4535-9560-c777e2008f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:55.582473 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee15588-c1df-4535-9560-c777e2008f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.582969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee15588-c1df-4535-9560-c777e2008f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.583401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee15588-c1df-4535-9560-c777e2008f24 admin admin] PATH_INFO: `/v3/users/5d6cd7874448444f9ff4a7637b161e0f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.584828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee15588-c1df-4535-9560-c777e2008f24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d6cd7874448444f9ff4a7637b161e0f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:55.587133 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee15588-c1df-4535-9560-c777e2008f24 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:55.596691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c172013-5685-4ec9-aeaa-a44a0ad17cd4 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:55.613674 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8570f-48a0-4522-8d5b-a9e23bc2ed80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:55.614770 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8570f-48a0-4522-8d5b-a9e23bc2ed80 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.615116 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8570f-48a0-4522-8d5b-a9e23bc2ed80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.615455 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8570f-48a0-4522-8d5b-a9e23bc2ed80 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.616252 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8570f-48a0-4522-8d5b-a9e23bc2ed80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:55.618360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8570f-48a0-4522-8d5b-a9e23bc2ed80 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:55.666954 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 983/1994] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:55 2022] POST /identity/v3/users => generated 290 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:55.667109 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6ee15588-c1df-4535-9560-c777e2008f24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d6cd7874448444f9ff4a7637b161e0f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:55.668591 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6ee15588-c1df-4535-9560-c777e2008f24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d6cd7874448444f9ff4a7637b161e0f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:55.669372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6ee15588-c1df-4535-9560-c777e2008f24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d6cd7874448444f9ff4a7637b161e0f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:55.672583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8570f-48a0-4522-8d5b-a9e23bc2ed80 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:55.674526 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4dd8570f-48a0-4522-8d5b-a9e23bc2ed80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:55.686287 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6ee15588-c1df-4535-9560-c777e2008f24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d6cd7874448444f9ff4a7637b161e0f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:55.687407 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8570f-48a0-4522-8d5b-a9e23bc2ed80 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:55.692671 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6c6a3-4af1-42a4-8217-86b9b1f4d0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:55.693340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6c6a3-4af1-42a4-8217-86b9b1f4d0a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.693621 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6c6a3-4af1-42a4-8217-86b9b1f4d0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.693901 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6c6a3-4af1-42a4-8217-86b9b1f4d0a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.694753 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6c6a3-4af1-42a4-8217-86b9b1f4d0a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:55.696404 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6c6a3-4af1-42a4-8217-86b9b1f4d0a7 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:55.696590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6ee15588-c1df-4535-9560-c777e2008f24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d6cd7874448444f9ff4a7637b161e0f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:55.707200 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 984/1995] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:55.708754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6c6a3-4af1-42a4-8217-86b9b1f4d0a7 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:55.710541 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-70b6c6a3-4af1-42a4-8217-86b9b1f4d0a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:55.713531 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1012/1996] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:55 2022] DELETE /identity/v3/users/5d6cd7874448444f9ff4a7637b161e0f => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:55.714527 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee15588-c1df-4535-9560-c777e2008f24 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:55.721279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:55.777488 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6c6a3-4af1-42a4-8217-86b9b1f4d0a7 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:55.830467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee15588-c1df-4535-9560-c777e2008f24 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:55.836294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46af49-f021-4aa7-88e2-e5e9ddc02254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:55.837212 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46af49-f021-4aa7-88e2-e5e9ddc02254 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.837496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46af49-f021-4aa7-88e2-e5e9ddc02254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.837775 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46af49-f021-4aa7-88e2-e5e9ddc02254 admin admin] PATH_INFO: `/v3/projects/da98156598eb407388e5210cc774e4fc/users/dfd8ab2feaca4d02a9ef4f1179d6d25f/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.849706 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb6f32-f291-4914-be95-c7d55e1f539d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:55.850912 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb6f32-f291-4914-be95-c7d55e1f539d service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.850912 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb6f32-f291-4914-be95-c7d55e1f539d service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.851239 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb6f32-f291-4914-be95-c7d55e1f539d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.853719 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e7bb6f32-f291-4914-be95-c7d55e1f539d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:55.855206 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46af49-f021-4aa7-88e2-e5e9ddc02254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da98156598eb407388e5210cc774e4fc, user_id=dfd8ab2feaca4d02a9ef4f1179d6d25f, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:55.857983 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46af49-f021-4aa7-88e2-e5e9ddc02254 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:55.880067 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1013/1997] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:55 2022] PUT /identity/v3/projects/da98156598eb407388e5210cc774e4fc/users/dfd8ab2feaca4d02a9ef4f1179d6d25f/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:55.881718 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1a793-0915-42a8-8ee0-db42cb5ce03c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.882016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1a793-0915-42a8-8ee0-db42cb5ce03c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.882302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1a793-0915-42a8-8ee0-db42cb5ce03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.903143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-21f1a793-0915-42a8-8ee0-db42cb5ce03c None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:55.904688 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-21f1a793-0915-42a8-8ee0-db42cb5ce03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:55.933535 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb6f32-f291-4914-be95-c7d55e1f539d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:55.936059 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb6f32-f291-4914-be95-c7d55e1f539d service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:55.948257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 985/1998] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:55.949205 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb6f32-f291-4914-be95-c7d55e1f539d service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:55.961471 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb6f32-f291-4914-be95-c7d55e1f539d service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:55.979927 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:55.987712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1014/1999] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:56.003892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1a793-0915-42a8-8ee0-db42cb5ce03c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:56.005808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-21f1a793-0915-42a8-8ee0-db42cb5ce03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:56.052824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09fbbc3-7cbf-4f2c-9c1d-697ef09ee14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:56.053524 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fbbc3-7cbf-4f2c-9c1d-697ef09ee14c admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:56.053799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fbbc3-7cbf-4f2c-9c1d-697ef09ee14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:56.054072 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fbbc3-7cbf-4f2c-9c1d-697ef09ee14c admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:56.056410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09fbbc3-7cbf-4f2c-9c1d-697ef09ee14c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:56.058832 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09fbbc3-7cbf-4f2c-9c1d-697ef09ee14c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:56.077920 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 986/2000] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:56.081907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09fbbc3-7cbf-4f2c-9c1d-697ef09ee14c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:56.093360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09fbbc3-7cbf-4f2c-9c1d-697ef09ee14c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:56.098701 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1a793-0915-42a8-8ee0-db42cb5ce03c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:56.099628 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ffda48-36e7-45a7-a818-aa17a88cb88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:56.100904 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffda48-36e7-45a7-a818-aa17a88cb88d admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:56.101543 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffda48-36e7-45a7-a818-aa17a88cb88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:56.102464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffda48-36e7-45a7-a818-aa17a88cb88d admin admin] PATH_INFO: `/v3/projects/da98156598eb407388e5210cc774e4fc/users/dfd8ab2feaca4d02a9ef4f1179d6d25f/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:56.102872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6602832-aa35-4a6c-923a-d479401ff935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:56.103669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6602832-aa35-4a6c-923a-d479401ff935 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:56.103999 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6602832-aa35-4a6c-923a-d479401ff935 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:56.104291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6602832-aa35-4a6c-923a-d479401ff935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:56.106073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ffda48-36e7-45a7-a818-aa17a88cb88d admin admin] RBAC: Authorizing `identity:create_grant(project_id=da98156598eb407388e5210cc774e4fc, user_id=dfd8ab2feaca4d02a9ef4f1179d6d25f, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:56.109865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ffda48-36e7-45a7-a818-aa17a88cb88d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:56.116040 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6602832-aa35-4a6c-923a-d479401ff935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:56.117893 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6602832-aa35-4a6c-923a-d479401ff935 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:56.128666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 987/2001] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:56 2022] PUT /identity/v3/projects/da98156598eb407388e5210cc774e4fc/users/dfd8ab2feaca4d02a9ef4f1179d6d25f/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:56.141419 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e03ca4-4efc-456a-a797-d31b14fb2b22 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:56.141704 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e03ca4-4efc-456a-a797-d31b14fb2b22 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:56.142179 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e03ca4-4efc-456a-a797-d31b14fb2b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:56.142603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1015/2002] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:16:56.174152 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-68e03ca4-4efc-456a-a797-d31b14fb2b22 None None] MFA Rules not processed for user `dfd8ab2feaca4d02a9ef4f1179d6d25f`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:56.175414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-68e03ca4-4efc-456a-a797-d31b14fb2b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:56.246198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 988/2003] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:56 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:56.257818 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6602832-aa35-4a6c-923a-d479401ff935 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:56.295296 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:56.321868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6602832-aa35-4a6c-923a-d479401ff935 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:56.382367 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b36d5b-0912-4f93-a5fa-12719bd69ab5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:56.383137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b36d5b-0912-4f93-a5fa-12719bd69ab5 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:56.383333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b36d5b-0912-4f93-a5fa-12719bd69ab5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:56.384211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b36d5b-0912-4f93-a5fa-12719bd69ab5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:56.399167 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b36d5b-0912-4f93-a5fa-12719bd69ab5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:56.400703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b36d5b-0912-4f93-a5fa-12719bd69ab5 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:56.438301 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1016/2004] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:56 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:56.497892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e03ca4-4efc-456a-a797-d31b14fb2b22 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:56.499625 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-68e03ca4-4efc-456a-a797-d31b14fb2b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:56.519319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e03ca4-4efc-456a-a797-d31b14fb2b22 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:56.526616 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb728329-546c-422e-aae5-209b1e28fe8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:56.527299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb728329-546c-422e-aae5-209b1e28fe8a service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:56.527586 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb728329-546c-422e-aae5-209b1e28fe8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:56.528188 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb728329-546c-422e-aae5-209b1e28fe8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:56.531415 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fb728329-546c-422e-aae5-209b1e28fe8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:56.649794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b36d5b-0912-4f93-a5fa-12719bd69ab5 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:56.651949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-35b36d5b-0912-4f93-a5fa-12719bd69ab5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:56.683890 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb728329-546c-422e-aae5-209b1e28fe8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:56.686252 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb728329-546c-422e-aae5-209b1e28fe8a service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:56.702313 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 989/2005] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:56 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:56.804724 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b36d5b-0912-4f93-a5fa-12719bd69ab5 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:56.815951 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071d0f8-1681-4952-8a5b-c057d0c104e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:56.816935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071d0f8-1681-4952-8a5b-c057d0c104e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:56.817197 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071d0f8-1681-4952-8a5b-c057d0c104e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:56.817465 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071d0f8-1681-4952-8a5b-c057d0c104e5 admin admin] PATH_INFO: `/v3/projects/52d2c8634d374385a59c1ffe45c0c95e` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:56.820211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071d0f8-1681-4952-8a5b-c057d0c104e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52d2c8634d374385a59c1ffe45c0c95e)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:56.822418 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071d0f8-1681-4952-8a5b-c057d0c104e5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:56.898145 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e071d0f8-1681-4952-8a5b-c057d0c104e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52d2c8634d374385a59c1ffe45c0c95e', 'request_id': 'req-e071d0f8-1681-4952-8a5b-c057d0c104e5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:56.912094 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e071d0f8-1681-4952-8a5b-c057d0c104e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52d2c8634d374385a59c1ffe45c0c95e', 'request_id': 'req-e071d0f8-1681-4952-8a5b-c057d0c104e5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:56.919781 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e071d0f8-1681-4952-8a5b-c057d0c104e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52d2c8634d374385a59c1ffe45c0c95e', 'request_id': 'req-e071d0f8-1681-4952-8a5b-c057d0c104e5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:56.934156 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e071d0f8-1681-4952-8a5b-c057d0c104e5 admin admin] The token cache is being invalidate because project 52d2c8634d374385a59c1ffe45c0c95e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:56.934156 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e071d0f8-1681-4952-8a5b-c057d0c104e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:56.936669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e071d0f8-1681-4952-8a5b-c057d0c104e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:56.939238 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1017/2006] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:56 2022] DELETE /identity/v3/projects/52d2c8634d374385a59c1ffe45c0c95e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:56.943413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb728329-546c-422e-aae5-209b1e28fe8a service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:56.971907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb728329-546c-422e-aae5-209b1e28fe8a service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:57.000818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:57.054097 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565acfa5-18af-467d-a84c-1c8c7c351ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:57.054893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565acfa5-18af-467d-a84c-1c8c7c351ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:57.055240 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565acfa5-18af-467d-a84c-1c8c7c351ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:57.055497 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565acfa5-18af-467d-a84c-1c8c7c351ca2 admin admin] PATH_INFO: `/v3/users/b41534f825ae4ca7851cd2e703b74bac` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:57.058063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565acfa5-18af-467d-a84c-1c8c7c351ca2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b41534f825ae4ca7851cd2e703b74bac)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:57.060510 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565acfa5-18af-467d-a84c-1c8c7c351ca2 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:57.133069 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-565acfa5-18af-467d-a84c-1c8c7c351ca2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b41534f825ae4ca7851cd2e703b74bac'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:57.133823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-565acfa5-18af-467d-a84c-1c8c7c351ca2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b41534f825ae4ca7851cd2e703b74bac'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:57.134417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-565acfa5-18af-467d-a84c-1c8c7c351ca2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b41534f825ae4ca7851cd2e703b74bac'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:57.165107 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-565acfa5-18af-467d-a84c-1c8c7c351ca2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b41534f825ae4ca7851cd2e703b74bac'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:57.192642 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-565acfa5-18af-467d-a84c-1c8c7c351ca2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b41534f825ae4ca7851cd2e703b74bac'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:57.222805 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 990/2007] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:56 2022] DELETE /identity/v3/users/b41534f825ae4ca7851cd2e703b74bac => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:57.733353 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071d0f8-1681-4952-8a5b-c057d0c104e5 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:57.735008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e071d0f8-1681-4952-8a5b-c057d0c104e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:57.749641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071d0f8-1681-4952-8a5b-c057d0c104e5 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:57.777610 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:57.849968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba032c-ab9d-4262-bc42-3ab738d963e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:57.851161 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba032c-ab9d-4262-bc42-3ab738d963e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:57.851651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba032c-ab9d-4262-bc42-3ab738d963e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:57.852097 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba032c-ab9d-4262-bc42-3ab738d963e8 admin admin] PATH_INFO: `/v3/projects/e8363f2d11654c95b32386104a420b16` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:57.853555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba032c-ab9d-4262-bc42-3ab738d963e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8363f2d11654c95b32386104a420b16)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:57.855508 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba032c-ab9d-4262-bc42-3ab738d963e8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:57.929074 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-feba032c-ab9d-4262-bc42-3ab738d963e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8363f2d11654c95b32386104a420b16', 'request_id': 'req-feba032c-ab9d-4262-bc42-3ab738d963e8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:57.939463 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-feba032c-ab9d-4262-bc42-3ab738d963e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8363f2d11654c95b32386104a420b16', 'request_id': 'req-feba032c-ab9d-4262-bc42-3ab738d963e8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:57.947083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-feba032c-ab9d-4262-bc42-3ab738d963e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8363f2d11654c95b32386104a420b16', 'request_id': 'req-feba032c-ab9d-4262-bc42-3ab738d963e8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:57.961278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-feba032c-ab9d-4262-bc42-3ab738d963e8 admin admin] The token cache is being invalidate because project e8363f2d11654c95b32386104a420b16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:57.961572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-feba032c-ab9d-4262-bc42-3ab738d963e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:57.962269 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-feba032c-ab9d-4262-bc42-3ab738d963e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:57.969706 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a2d5a-ca4c-4e74-b49b-e8aeafec11b3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:57.970026 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a2d5a-ca4c-4e74-b49b-e8aeafec11b3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:57.970304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a2d5a-ca4c-4e74-b49b-e8aeafec11b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:57.972444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1018/2008] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:57 2022] DELETE /identity/v3/projects/e8363f2d11654c95b32386104a420b16 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:58.002448 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6a4a2d5a-ca4c-4e74-b49b-e8aeafec11b3 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:58.004082 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6a4a2d5a-ca4c-4e74-b49b-e8aeafec11b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:58.083562 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51541e45-5d62-4b0f-951b-821af3d3d11b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:58.083865 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51541e45-5d62-4b0f-951b-821af3d3d11b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:58.084394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51541e45-5d62-4b0f-951b-821af3d3d11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:58.087749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 991/2009] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:58.103119 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-51541e45-5d62-4b0f-951b-821af3d3d11b None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:58.104379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-51541e45-5d62-4b0f-951b-821af3d3d11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:58.116593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1019/2010] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:58.118384 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a2d5a-ca4c-4e74-b49b-e8aeafec11b3 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:58.135216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a2d5a-ca4c-4e74-b49b-e8aeafec11b3 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:58.148037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707602ee-08e5-489e-900a-779edabc548e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:58.149749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707602ee-08e5-489e-900a-779edabc548e admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:58.150292 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707602ee-08e5-489e-900a-779edabc548e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:58.150738 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707602ee-08e5-489e-900a-779edabc548e admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:58.151737 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707602ee-08e5-489e-900a-779edabc548e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:58.152327 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707602ee-08e5-489e-900a-779edabc548e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:58.154490 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707602ee-08e5-489e-900a-779edabc548e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:58.168222 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 992/2011] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:16:58.174413 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51541e45-5d62-4b0f-951b-821af3d3d11b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:58.185491 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51541e45-5d62-4b0f-951b-821af3d3d11b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:58.197456 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1122e-1ed2-48ba-8eb5-86f623b1af6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:58.198156 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1122e-1ed2-48ba-8eb5-86f623b1af6a admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:58.199234 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1122e-1ed2-48ba-8eb5-86f623b1af6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:58.199234 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1122e-1ed2-48ba-8eb5-86f623b1af6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:58.199448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1122e-1ed2-48ba-8eb5-86f623b1af6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:58.202166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1122e-1ed2-48ba-8eb5-86f623b1af6a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:58.229667 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1020/2012] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:58 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:58.233846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707602ee-08e5-489e-900a-779edabc548e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:58.244233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707602ee-08e5-489e-900a-779edabc548e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:58.274424 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:58.307288 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1dac0-9897-4bbd-9f70-1cb10a405522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:58.308011 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1dac0-9897-4bbd-9f70-1cb10a405522 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:58.308293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1dac0-9897-4bbd-9f70-1cb10a405522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:58.308570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1dac0-9897-4bbd-9f70-1cb10a405522 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:58.309306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d1dac0-9897-4bbd-9f70-1cb10a405522 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:58.311636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d1dac0-9897-4bbd-9f70-1cb10a405522 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:58.341318 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 993/2013] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:58 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:58.345288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1122e-1ed2-48ba-8eb5-86f623b1af6a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:58.355906 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1122e-1ed2-48ba-8eb5-86f623b1af6a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:58.361993 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cc45a-8902-4278-8b3c-a99350ab6f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:58.363134 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cc45a-8902-4278-8b3c-a99350ab6f9b admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:58.363134 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cc45a-8902-4278-8b3c-a99350ab6f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:58.363692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cc45a-8902-4278-8b3c-a99350ab6f9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:58.364735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56cc45a-8902-4278-8b3c-a99350ab6f9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:58.367205 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56cc45a-8902-4278-8b3c-a99350ab6f9b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:58.380402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1021/2014] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:58.386810 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1dac0-9897-4bbd-9f70-1cb10a405522 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:58.397760 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1dac0-9897-4bbd-9f70-1cb10a405522 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:58.402741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d910c2-6b12-4494-83b6-2b2d1dd89e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:58.404032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d910c2-6b12-4494-83b6-2b2d1dd89e4f admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:58.404032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d910c2-6b12-4494-83b6-2b2d1dd89e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:58.404288 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d910c2-6b12-4494-83b6-2b2d1dd89e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:58.405090 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d910c2-6b12-4494-83b6-2b2d1dd89e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:58.406759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d910c2-6b12-4494-83b6-2b2d1dd89e4f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:58.421671 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 994/2015] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:58.426027 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cc45a-8902-4278-8b3c-a99350ab6f9b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:58.440803 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cc45a-8902-4278-8b3c-a99350ab6f9b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:58.445792 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345efce8-862c-4c1b-85b3-0864d7ac753a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:58.446849 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345efce8-862c-4c1b-85b3-0864d7ac753a admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:58.447187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345efce8-862c-4c1b-85b3-0864d7ac753a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:58.447599 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345efce8-862c-4c1b-85b3-0864d7ac753a admin admin] PATH_INFO: `/v3/projects/e92d327de86441be99184405591568f7/users/67209d464f77471c8ccf9ef3baf95437/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:58.470150 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345efce8-862c-4c1b-85b3-0864d7ac753a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e92d327de86441be99184405591568f7, user_id=67209d464f77471c8ccf9ef3baf95437, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:58.472964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345efce8-862c-4c1b-85b3-0864d7ac753a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:58.495186 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1022/2016] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:58 2022] PUT /identity/v3/projects/e92d327de86441be99184405591568f7/users/67209d464f77471c8ccf9ef3baf95437/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:58.505425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3515f7d-95b6-4b06-8669-cf06b489cbcb None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:58.505716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3515f7d-95b6-4b06-8669-cf06b489cbcb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:58.505995 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3515f7d-95b6-4b06-8669-cf06b489cbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:58.534653 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a3515f7d-95b6-4b06-8669-cf06b489cbcb None None] MFA Rules not processed for user `67209d464f77471c8ccf9ef3baf95437`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:58.536058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a3515f7d-95b6-4b06-8669-cf06b489cbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:58.619751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 995/2017] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:16:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:58.691641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345efce8-862c-4c1b-85b3-0864d7ac753a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:58.694114 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-345efce8-862c-4c1b-85b3-0864d7ac753a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:58.793492 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:58.804286 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345efce8-862c-4c1b-85b3-0864d7ac753a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:58.810023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50446d5-abe0-47d8-bc78-f5a375c6d817 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:58.810388 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50446d5-abe0-47d8-bc78-f5a375c6d817 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:58.810738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50446d5-abe0-47d8-bc78-f5a375c6d817 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:58.811086 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50446d5-abe0-47d8-bc78-f5a375c6d817 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:58.832860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50446d5-abe0-47d8-bc78-f5a375c6d817 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:58.834361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50446d5-abe0-47d8-bc78-f5a375c6d817 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:58.851839 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1023/2018] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:00.238424 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3515f7d-95b6-4b06-8669-cf06b489cbcb None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:00.241063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a3515f7d-95b6-4b06-8669-cf06b489cbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:00.308534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3515f7d-95b6-4b06-8669-cf06b489cbcb None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:00.313325 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c510af1-bed9-447c-8208-edd2ebe8db6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:00.314215 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c510af1-bed9-447c-8208-edd2ebe8db6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:00.314496 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c510af1-bed9-447c-8208-edd2ebe8db6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:00.314766 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c510af1-bed9-447c-8208-edd2ebe8db6c admin admin] PATH_INFO: `/v3/users/9234efa022e54c828195abea4dd67acf` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:00.316027 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c510af1-bed9-447c-8208-edd2ebe8db6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9234efa022e54c828195abea4dd67acf)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:00.317951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c510af1-bed9-447c-8208-edd2ebe8db6c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:00.367288 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:00.375170 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1c510af1-bed9-447c-8208-edd2ebe8db6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9234efa022e54c828195abea4dd67acf'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:00.376024 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1c510af1-bed9-447c-8208-edd2ebe8db6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9234efa022e54c828195abea4dd67acf'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:00.376772 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1c510af1-bed9-447c-8208-edd2ebe8db6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9234efa022e54c828195abea4dd67acf'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:00.395113 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1c510af1-bed9-447c-8208-edd2ebe8db6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9234efa022e54c828195abea4dd67acf'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:00.404170 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1c510af1-bed9-447c-8208-edd2ebe8db6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9234efa022e54c828195abea4dd67acf'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:00.420728 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 996/2019] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:00 2022] DELETE /identity/v3/users/9234efa022e54c828195abea4dd67acf => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:00.425215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c086ac-b8ad-4a29-8e95-0811efc05411 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:00.425723 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c086ac-b8ad-4a29-8e95-0811efc05411 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:00.425813 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c086ac-b8ad-4a29-8e95-0811efc05411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:00.444046 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e2c086ac-b8ad-4a29-8e95-0811efc05411 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:00.445585 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e2c086ac-b8ad-4a29-8e95-0811efc05411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:00.514539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1024/2020] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:00.524860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c510af1-bed9-447c-8208-edd2ebe8db6c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:00.526452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1c510af1-bed9-447c-8208-edd2ebe8db6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:00.602737 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c510af1-bed9-447c-8208-edd2ebe8db6c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:00.607275 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cfcd7-984b-4a3b-8435-0efe42fc42f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:00.608495 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cfcd7-984b-4a3b-8435-0efe42fc42f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:00.609022 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cfcd7-984b-4a3b-8435-0efe42fc42f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:00.609368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cfcd7-984b-4a3b-8435-0efe42fc42f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:00.622118 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90cfcd7-984b-4a3b-8435-0efe42fc42f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:00.624529 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90cfcd7-984b-4a3b-8435-0efe42fc42f6 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:00.638526 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 997/2021] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:01.009974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c086ac-b8ad-4a29-8e95-0811efc05411 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:01.011748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e2c086ac-b8ad-4a29-8e95-0811efc05411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:01.061843 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c086ac-b8ad-4a29-8e95-0811efc05411 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:01.090676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f75cb-3609-4ed2-b379-b2f428cff9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:01.091702 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f75cb-3609-4ed2-b379-b2f428cff9d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:01.092157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f75cb-3609-4ed2-b379-b2f428cff9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:01.092494 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f75cb-3609-4ed2-b379-b2f428cff9d2 admin admin] PATH_INFO: `/v3/projects/ae1696d8e86649b88a1aefb4e8614f8b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:01.102924 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f75cb-3609-4ed2-b379-b2f428cff9d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae1696d8e86649b88a1aefb4e8614f8b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:01.116972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f75cb-3609-4ed2-b379-b2f428cff9d2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:01.217371 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a75f75cb-3609-4ed2-b379-b2f428cff9d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae1696d8e86649b88a1aefb4e8614f8b', 'request_id': 'req-a75f75cb-3609-4ed2-b379-b2f428cff9d2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:01.226881 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a75f75cb-3609-4ed2-b379-b2f428cff9d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae1696d8e86649b88a1aefb4e8614f8b', 'request_id': 'req-a75f75cb-3609-4ed2-b379-b2f428cff9d2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:01.233824 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a75f75cb-3609-4ed2-b379-b2f428cff9d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae1696d8e86649b88a1aefb4e8614f8b', 'request_id': 'req-a75f75cb-3609-4ed2-b379-b2f428cff9d2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:01.248397 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a75f75cb-3609-4ed2-b379-b2f428cff9d2 admin admin] The token cache is being invalidate because project ae1696d8e86649b88a1aefb4e8614f8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:01.248969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a75f75cb-3609-4ed2-b379-b2f428cff9d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:01.249986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a75f75cb-3609-4ed2-b379-b2f428cff9d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:01.251814 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1025/2022] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:01 2022] DELETE /identity/v3/projects/ae1696d8e86649b88a1aefb4e8614f8b => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:01.256604 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cfcd7-984b-4a3b-8435-0efe42fc42f6 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:01.268795 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cfcd7-984b-4a3b-8435-0efe42fc42f6 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:01.343991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fddfd5-7598-4868-a06c-2d17fa6bbd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:01.345317 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fddfd5-7598-4868-a06c-2d17fa6bbd51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:01.345806 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fddfd5-7598-4868-a06c-2d17fa6bbd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:01.346295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fddfd5-7598-4868-a06c-2d17fa6bbd51 admin admin] PATH_INFO: `/v3/users/dfd8ab2feaca4d02a9ef4f1179d6d25f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:01.348363 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fddfd5-7598-4868-a06c-2d17fa6bbd51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfd8ab2feaca4d02a9ef4f1179d6d25f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:01.351121 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fddfd5-7598-4868-a06c-2d17fa6bbd51 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:01.405708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a9fddfd5-7598-4868-a06c-2d17fa6bbd51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfd8ab2feaca4d02a9ef4f1179d6d25f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:01.406282 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a9fddfd5-7598-4868-a06c-2d17fa6bbd51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfd8ab2feaca4d02a9ef4f1179d6d25f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:01.406886 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a9fddfd5-7598-4868-a06c-2d17fa6bbd51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfd8ab2feaca4d02a9ef4f1179d6d25f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:01.417735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:01.419330 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a9fddfd5-7598-4868-a06c-2d17fa6bbd51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfd8ab2feaca4d02a9ef4f1179d6d25f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:01.428273 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a9fddfd5-7598-4868-a06c-2d17fa6bbd51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfd8ab2feaca4d02a9ef4f1179d6d25f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:01.446015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 998/2023] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:01 2022] DELETE /identity/v3/users/dfd8ab2feaca4d02a9ef4f1179d6d25f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:01.730011 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f75cb-3609-4ed2-b379-b2f428cff9d2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:01.731507 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a75f75cb-3609-4ed2-b379-b2f428cff9d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:01.751696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f75cb-3609-4ed2-b379-b2f428cff9d2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:01.812715 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66315273-9f05-4392-8e7f-a53f9617eb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:01.813582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66315273-9f05-4392-8e7f-a53f9617eb4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:01.813859 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66315273-9f05-4392-8e7f-a53f9617eb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:01.814129 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66315273-9f05-4392-8e7f-a53f9617eb4d admin admin] PATH_INFO: `/v3/projects/da98156598eb407388e5210cc774e4fc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:01.815363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66315273-9f05-4392-8e7f-a53f9617eb4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=da98156598eb407388e5210cc774e4fc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:01.817252 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66315273-9f05-4392-8e7f-a53f9617eb4d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:01.833044 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:01.866824 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-66315273-9f05-4392-8e7f-a53f9617eb4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da98156598eb407388e5210cc774e4fc', 'request_id': 'req-66315273-9f05-4392-8e7f-a53f9617eb4d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:01.876136 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-66315273-9f05-4392-8e7f-a53f9617eb4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da98156598eb407388e5210cc774e4fc', 'request_id': 'req-66315273-9f05-4392-8e7f-a53f9617eb4d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:01.881563 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-66315273-9f05-4392-8e7f-a53f9617eb4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da98156598eb407388e5210cc774e4fc', 'request_id': 'req-66315273-9f05-4392-8e7f-a53f9617eb4d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:01.893308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-66315273-9f05-4392-8e7f-a53f9617eb4d admin admin] The token cache is being invalidate because project da98156598eb407388e5210cc774e4fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:01.893569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-66315273-9f05-4392-8e7f-a53f9617eb4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:01.894349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-66315273-9f05-4392-8e7f-a53f9617eb4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:01.896184 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1026/2024] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:01 2022] DELETE /identity/v3/projects/da98156598eb407388e5210cc774e4fc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:01.901276 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790e4bb-9bae-4637-bc84-c4619393c43a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:01.901618 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790e4bb-9bae-4637-bc84-c4619393c43a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:01.901898 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790e4bb-9bae-4637-bc84-c4619393c43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:01.920975 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5790e4bb-9bae-4637-bc84-c4619393c43a None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:01.922263 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5790e4bb-9bae-4637-bc84-c4619393c43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:02.001613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 999/2025] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:02.009243 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e4f16-e194-4208-8c52-02bf35275e9d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:02.009528 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e4f16-e194-4208-8c52-02bf35275e9d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:02.009799 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e4f16-e194-4208-8c52-02bf35275e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:02.029117 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-863e4f16-e194-4208-8c52-02bf35275e9d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:02.030325 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-863e4f16-e194-4208-8c52-02bf35275e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:02.038191 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1027/2026] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:02.042752 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790e4bb-9bae-4637-bc84-c4619393c43a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:02.053187 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790e4bb-9bae-4637-bc84-c4619393c43a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:02.058482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0dbe12-b13c-413d-9d69-9f9e24365283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:02.059172 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0dbe12-b13c-413d-9d69-9f9e24365283 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:02.059456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0dbe12-b13c-413d-9d69-9f9e24365283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:02.059905 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0dbe12-b13c-413d-9d69-9f9e24365283 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:02.060672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0dbe12-b13c-413d-9d69-9f9e24365283 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:02.060982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0dbe12-b13c-413d-9d69-9f9e24365283 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:02.062651 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0dbe12-b13c-413d-9d69-9f9e24365283 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:02.074804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1000/2027] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:17:02.079432 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e4f16-e194-4208-8c52-02bf35275e9d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:02.089862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e4f16-e194-4208-8c52-02bf35275e9d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:02.095383 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99270c7-68d4-489a-9287-c424f0d9ca65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:02.096158 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99270c7-68d4-489a-9287-c424f0d9ca65 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:02.096437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99270c7-68d4-489a-9287-c424f0d9ca65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:02.096700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99270c7-68d4-489a-9287-c424f0d9ca65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:02.097385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99270c7-68d4-489a-9287-c424f0d9ca65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:02.099291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99270c7-68d4-489a-9287-c424f0d9ca65 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:02.128100 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1028/2028] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:02 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:02.132255 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0dbe12-b13c-413d-9d69-9f9e24365283 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:02.143632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0dbe12-b13c-413d-9d69-9f9e24365283 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:02.207726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a48471-46d5-4cfb-b599-7b8ee02a48d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:02.208400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a48471-46d5-4cfb-b599-7b8ee02a48d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:02.208710 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a48471-46d5-4cfb-b599-7b8ee02a48d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:02.208983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a48471-46d5-4cfb-b599-7b8ee02a48d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:02.209686 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a48471-46d5-4cfb-b599-7b8ee02a48d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:02.211704 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a48471-46d5-4cfb-b599-7b8ee02a48d1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:02.231655 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1001/2029] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:02 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:02.235734 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99270c7-68d4-489a-9287-c424f0d9ca65 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:02.246369 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99270c7-68d4-489a-9287-c424f0d9ca65 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:02.251155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26ce0d-c4b3-428a-a5c0-0cb8fc7464d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:02.251854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26ce0d-c4b3-428a-a5c0-0cb8fc7464d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:02.252159 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26ce0d-c4b3-428a-a5c0-0cb8fc7464d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:02.252431 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26ce0d-c4b3-428a-a5c0-0cb8fc7464d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:02.253188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26ce0d-c4b3-428a-a5c0-0cb8fc7464d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:02.254824 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26ce0d-c4b3-428a-a5c0-0cb8fc7464d4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:02.265704 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1029/2030] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:02.270031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a48471-46d5-4cfb-b599-7b8ee02a48d1 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:02.280644 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a48471-46d5-4cfb-b599-7b8ee02a48d1 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:02.285849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f561f22-dd77-462c-90dd-0cce8ce7772d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:02.286518 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f561f22-dd77-462c-90dd-0cce8ce7772d admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:02.286799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f561f22-dd77-462c-90dd-0cce8ce7772d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:02.287057 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f561f22-dd77-462c-90dd-0cce8ce7772d admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:02.287836 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f561f22-dd77-462c-90dd-0cce8ce7772d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:02.289471 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f561f22-dd77-462c-90dd-0cce8ce7772d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:02.299914 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1002/2031] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:02.304203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26ce0d-c4b3-428a-a5c0-0cb8fc7464d4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:02.314420 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26ce0d-c4b3-428a-a5c0-0cb8fc7464d4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:02.319387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854e533-382d-4877-bcc1-1e8961434b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:02.320310 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854e533-382d-4877-bcc1-1e8961434b39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:02.320623 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854e533-382d-4877-bcc1-1e8961434b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:02.320896 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854e533-382d-4877-bcc1-1e8961434b39 admin admin] PATH_INFO: `/v3/projects/e9616eaa757b42dd9ee9871fbb99592b/users/115aae19ff1f4f8ea879b98fd4e2c701/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:02.337222 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8854e533-382d-4877-bcc1-1e8961434b39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9616eaa757b42dd9ee9871fbb99592b, user_id=115aae19ff1f4f8ea879b98fd4e2c701, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:02.340038 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8854e533-382d-4877-bcc1-1e8961434b39 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:02.355648 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1030/2032] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:02 2022] PUT /identity/v3/projects/e9616eaa757b42dd9ee9871fbb99592b/users/115aae19ff1f4f8ea879b98fd4e2c701/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:02.367001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d23d67-e3d8-40e3-8618-6dbc8760edb6 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:02.367324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d23d67-e3d8-40e3-8618-6dbc8760edb6 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:02.367639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d23d67-e3d8-40e3-8618-6dbc8760edb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:02.399725 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a9d23d67-e3d8-40e3-8618-6dbc8760edb6 None None] MFA Rules not processed for user `115aae19ff1f4f8ea879b98fd4e2c701`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:02.400932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a9d23d67-e3d8-40e3-8618-6dbc8760edb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:02.435821 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:02.473881 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1003/2033] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:02.475980 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854e533-382d-4877-bcc1-1e8961434b39 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:02.552524 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854e533-382d-4877-bcc1-1e8961434b39 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:02.560812 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433ce44-fc2a-4fa8-95c2-0e6f68765313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:02.561487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433ce44-fc2a-4fa8-95c2-0e6f68765313 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:02.561768 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433ce44-fc2a-4fa8-95c2-0e6f68765313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:02.562043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433ce44-fc2a-4fa8-95c2-0e6f68765313 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:02.562727 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433ce44-fc2a-4fa8-95c2-0e6f68765313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:02.564775 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433ce44-fc2a-4fa8-95c2-0e6f68765313 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:02.592365 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1031/2034] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:02 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:02.596501 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d23d67-e3d8-40e3-8618-6dbc8760edb6 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:02.607705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d23d67-e3d8-40e3-8618-6dbc8760edb6 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:02.665401 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5fd71-587a-4cca-8b59-9a2bcc47f465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:02.666110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5fd71-587a-4cca-8b59-9a2bcc47f465 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:02.666530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5fd71-587a-4cca-8b59-9a2bcc47f465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:02.666815 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5fd71-587a-4cca-8b59-9a2bcc47f465 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:02.667514 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5fd71-587a-4cca-8b59-9a2bcc47f465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:02.669487 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5fd71-587a-4cca-8b59-9a2bcc47f465 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:02.691849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1004/2035] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:02 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:02.695828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433ce44-fc2a-4fa8-95c2-0e6f68765313 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:02.706179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433ce44-fc2a-4fa8-95c2-0e6f68765313 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:02.710962 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a2a7c-bbf1-4bca-a88b-ef6ff575a340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:02.711642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a2a7c-bbf1-4bca-a88b-ef6ff575a340 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:02.711926 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a2a7c-bbf1-4bca-a88b-ef6ff575a340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:02.712196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a2a7c-bbf1-4bca-a88b-ef6ff575a340 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:02.713004 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a2a7c-bbf1-4bca-a88b-ef6ff575a340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:02.714596 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a2a7c-bbf1-4bca-a88b-ef6ff575a340 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:02.725227 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1032/2036] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:02.729276 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5fd71-587a-4cca-8b59-9a2bcc47f465 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:02.739375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5fd71-587a-4cca-8b59-9a2bcc47f465 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:02.744428 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90345cd-46b0-4f08-8c90-0c450e65dc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:02.745320 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90345cd-46b0-4f08-8c90-0c450e65dc0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:02.745607 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90345cd-46b0-4f08-8c90-0c450e65dc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:02.745889 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90345cd-46b0-4f08-8c90-0c450e65dc0a admin admin] PATH_INFO: `/v3/projects/d012ed4035ac4824b6454a26961c6bfa/users/ae98f71787884953a6f47dcbe506b187/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:02.761864 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90345cd-46b0-4f08-8c90-0c450e65dc0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d012ed4035ac4824b6454a26961c6bfa, user_id=ae98f71787884953a6f47dcbe506b187, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:02.764593 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90345cd-46b0-4f08-8c90-0c450e65dc0a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:02.779767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1005/2037] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:02 2022] PUT /identity/v3/projects/d012ed4035ac4824b6454a26961c6bfa/users/ae98f71787884953a6f47dcbe506b187/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:02.783803 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a2a7c-bbf1-4bca-a88b-ef6ff575a340 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:02.794272 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a2a7c-bbf1-4bca-a88b-ef6ff575a340 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:02.803410 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:02.852920 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a0a49-36b2-40f2-8298-e85647674ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:02.853589 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a0a49-36b2-40f2-8298-e85647674ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:02.853868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a0a49-36b2-40f2-8298-e85647674ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:02.854140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a0a49-36b2-40f2-8298-e85647674ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:02.854862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a0a49-36b2-40f2-8298-e85647674ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:02.856588 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a0a49-36b2-40f2-8298-e85647674ef1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:02.866671 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1033/2038] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:02.870810 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90345cd-46b0-4f08-8c90-0c450e65dc0a admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:02.881508 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90345cd-46b0-4f08-8c90-0c450e65dc0a admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:02.886454 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61c5ec-efd2-4c38-a3a1-c457f67e527d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:02.887350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61c5ec-efd2-4c38-a3a1-c457f67e527d admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:02.887891 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61c5ec-efd2-4c38-a3a1-c457f67e527d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:02.888166 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61c5ec-efd2-4c38-a3a1-c457f67e527d admin admin] PATH_INFO: `/v3/projects/d012ed4035ac4824b6454a26961c6bfa/users/ae98f71787884953a6f47dcbe506b187/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:02.890237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61c5ec-efd2-4c38-a3a1-c457f67e527d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d012ed4035ac4824b6454a26961c6bfa, user_id=ae98f71787884953a6f47dcbe506b187, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:02.893216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61c5ec-efd2-4c38-a3a1-c457f67e527d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:02.904073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1006/2039] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:02 2022] PUT /identity/v3/projects/d012ed4035ac4824b6454a26961c6bfa/users/ae98f71787884953a6f47dcbe506b187/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:02.914133 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0713749-2bfb-4aca-9ca6-18f4ea9f1386 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:02.914416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0713749-2bfb-4aca-9ca6-18f4ea9f1386 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:02.914689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0713749-2bfb-4aca-9ca6-18f4ea9f1386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:02.944172 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f0713749-2bfb-4aca-9ca6-18f4ea9f1386 None None] MFA Rules not processed for user `ae98f71787884953a6f47dcbe506b187`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:02.945487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f0713749-2bfb-4aca-9ca6-18f4ea9f1386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:03.017922 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1034/2040] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:03.029172 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61c5ec-efd2-4c38-a3a1-c457f67e527d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:03.031051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9e61c5ec-efd2-4c38-a3a1-c457f67e527d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:03.110246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61c5ec-efd2-4c38-a3a1-c457f67e527d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:03.114836 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae9d4e-0f85-42d7-b6cb-03990f02e3cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:03.115500 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae9d4e-0f85-42d7-b6cb-03990f02e3cb service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:03.115822 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae9d4e-0f85-42d7-b6cb-03990f02e3cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:03.116101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae9d4e-0f85-42d7-b6cb-03990f02e3cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:03.126300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae9d4e-0f85-42d7-b6cb-03990f02e3cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:03.127785 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae9d4e-0f85-42d7-b6cb-03990f02e3cb service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:03.137489 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1007/2041] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:03.683285 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0713749-2bfb-4aca-9ca6-18f4ea9f1386 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:03.757711 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0713749-2bfb-4aca-9ca6-18f4ea9f1386 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:03.763113 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bab525-9733-4c21-ba86-51f3fdb2f8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:03.764081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bab525-9733-4c21-ba86-51f3fdb2f8cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:03.764367 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bab525-9733-4c21-ba86-51f3fdb2f8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:03.764651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bab525-9733-4c21-ba86-51f3fdb2f8cb admin admin] PATH_INFO: `/v3/users/115aae19ff1f4f8ea879b98fd4e2c701` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:03.765921 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bab525-9733-4c21-ba86-51f3fdb2f8cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=115aae19ff1f4f8ea879b98fd4e2c701)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:03.768232 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bab525-9733-4c21-ba86-51f3fdb2f8cb admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:03.808658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:03.817778 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d5bab525-9733-4c21-ba86-51f3fdb2f8cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '115aae19ff1f4f8ea879b98fd4e2c701'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:03.818419 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d5bab525-9733-4c21-ba86-51f3fdb2f8cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '115aae19ff1f4f8ea879b98fd4e2c701'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:03.819014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d5bab525-9733-4c21-ba86-51f3fdb2f8cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '115aae19ff1f4f8ea879b98fd4e2c701'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:03.830589 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d5bab525-9733-4c21-ba86-51f3fdb2f8cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '115aae19ff1f4f8ea879b98fd4e2c701'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:03.837507 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d5bab525-9733-4c21-ba86-51f3fdb2f8cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '115aae19ff1f4f8ea879b98fd4e2c701'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:03.852666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1035/2042] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:03 2022] DELETE /identity/v3/users/115aae19ff1f4f8ea879b98fd4e2c701 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:03.857301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210a1f8-dca4-4430-a839-eb27d1ebd3c4 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:03.857723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210a1f8-dca4-4430-a839-eb27d1ebd3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:03.857841 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210a1f8-dca4-4430-a839-eb27d1ebd3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:03.876379 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-2210a1f8-dca4-4430-a839-eb27d1ebd3c4 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:03.877610 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2210a1f8-dca4-4430-a839-eb27d1ebd3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:03.905468 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:03.942724 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1008/2043] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:03.954783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bab525-9733-4c21-ba86-51f3fdb2f8cb admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:03.956411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d5bab525-9733-4c21-ba86-51f3fdb2f8cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:04.020182 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bab525-9733-4c21-ba86-51f3fdb2f8cb admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:04.024199 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00e57d-ec4a-4fd3-8f83-dbbd8ec42922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:04.024854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00e57d-ec4a-4fd3-8f83-dbbd8ec42922 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:04.025136 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00e57d-ec4a-4fd3-8f83-dbbd8ec42922 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:04.025411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00e57d-ec4a-4fd3-8f83-dbbd8ec42922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:04.035039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00e57d-ec4a-4fd3-8f83-dbbd8ec42922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:04.036519 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00e57d-ec4a-4fd3-8f83-dbbd8ec42922 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:04.046478 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1036/2044] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:04.324761 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210a1f8-dca4-4430-a839-eb27d1ebd3c4 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:04.326588 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2210a1f8-dca4-4430-a839-eb27d1ebd3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:04.340805 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210a1f8-dca4-4430-a839-eb27d1ebd3c4 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:04.346007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc1c0e-2d99-4099-8c35-afcca71c95c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:04.346907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc1c0e-2d99-4099-8c35-afcca71c95c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:04.347190 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc1c0e-2d99-4099-8c35-afcca71c95c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:04.347462 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc1c0e-2d99-4099-8c35-afcca71c95c1 admin admin] PATH_INFO: `/v3/projects/e9616eaa757b42dd9ee9871fbb99592b` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:04.348910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc1c0e-2d99-4099-8c35-afcca71c95c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9616eaa757b42dd9ee9871fbb99592b)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:04.350878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc1c0e-2d99-4099-8c35-afcca71c95c1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:04.400652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e8fc1c0e-2d99-4099-8c35-afcca71c95c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9616eaa757b42dd9ee9871fbb99592b', 'request_id': 'req-e8fc1c0e-2d99-4099-8c35-afcca71c95c1'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.409790 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e8fc1c0e-2d99-4099-8c35-afcca71c95c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9616eaa757b42dd9ee9871fbb99592b', 'request_id': 'req-e8fc1c0e-2d99-4099-8c35-afcca71c95c1'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.416149 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e8fc1c0e-2d99-4099-8c35-afcca71c95c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9616eaa757b42dd9ee9871fbb99592b', 'request_id': 'req-e8fc1c0e-2d99-4099-8c35-afcca71c95c1'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.427430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e8fc1c0e-2d99-4099-8c35-afcca71c95c1 admin admin] The token cache is being invalidate because project e9616eaa757b42dd9ee9871fbb99592b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:04.427740 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e8fc1c0e-2d99-4099-8c35-afcca71c95c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.428435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e8fc1c0e-2d99-4099-8c35-afcca71c95c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.430056 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1009/2045] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:04 2022] DELETE /identity/v3/projects/e9616eaa757b42dd9ee9871fbb99592b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:04.435089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00e57d-ec4a-4fd3-8f83-dbbd8ec42922 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:04.446238 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00e57d-ec4a-4fd3-8f83-dbbd8ec42922 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:04.512489 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed243f4-3620-4282-a8da-e136efdabfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:04.513407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed243f4-3620-4282-a8da-e136efdabfdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:04.513694 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed243f4-3620-4282-a8da-e136efdabfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:04.513968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed243f4-3620-4282-a8da-e136efdabfdb admin admin] PATH_INFO: `/v3/users/ae98f71787884953a6f47dcbe506b187` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:04.515160 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed243f4-3620-4282-a8da-e136efdabfdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae98f71787884953a6f47dcbe506b187)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:04.517493 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed243f4-3620-4282-a8da-e136efdabfdb admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:04.569807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5ed243f4-3620-4282-a8da-e136efdabfdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae98f71787884953a6f47dcbe506b187'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.570440 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5ed243f4-3620-4282-a8da-e136efdabfdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae98f71787884953a6f47dcbe506b187'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.571103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5ed243f4-3620-4282-a8da-e136efdabfdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae98f71787884953a6f47dcbe506b187'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.579459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:04.584919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5ed243f4-3620-4282-a8da-e136efdabfdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae98f71787884953a6f47dcbe506b187'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.592895 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5ed243f4-3620-4282-a8da-e136efdabfdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae98f71787884953a6f47dcbe506b187'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.607048 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1037/2046] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:04 2022] DELETE /identity/v3/users/ae98f71787884953a6f47dcbe506b187 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:04.900163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc1c0e-2d99-4099-8c35-afcca71c95c1 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:04.901994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e8fc1c0e-2d99-4099-8c35-afcca71c95c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:04.917262 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc1c0e-2d99-4099-8c35-afcca71c95c1 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:04.985634 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6d5e0-87b0-4d05-a178-2637f4f2dc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:04.986554 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6d5e0-87b0-4d05-a178-2637f4f2dc18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:04.986866 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6d5e0-87b0-4d05-a178-2637f4f2dc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:04.987143 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6d5e0-87b0-4d05-a178-2637f4f2dc18 admin admin] PATH_INFO: `/v3/projects/d012ed4035ac4824b6454a26961c6bfa` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:04.988320 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6d5e0-87b0-4d05-a178-2637f4f2dc18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d012ed4035ac4824b6454a26961c6bfa)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:04.990260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6d5e0-87b0-4d05-a178-2637f4f2dc18 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:05.041955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-92e6d5e0-87b0-4d05-a178-2637f4f2dc18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd012ed4035ac4824b6454a26961c6bfa', 'request_id': 'req-92e6d5e0-87b0-4d05-a178-2637f4f2dc18'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:05.052184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-92e6d5e0-87b0-4d05-a178-2637f4f2dc18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd012ed4035ac4824b6454a26961c6bfa', 'request_id': 'req-92e6d5e0-87b0-4d05-a178-2637f4f2dc18'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:05.053855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:05.065058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-92e6d5e0-87b0-4d05-a178-2637f4f2dc18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd012ed4035ac4824b6454a26961c6bfa', 'request_id': 'req-92e6d5e0-87b0-4d05-a178-2637f4f2dc18'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:05.077889 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-92e6d5e0-87b0-4d05-a178-2637f4f2dc18 admin admin] The token cache is being invalidate because project d012ed4035ac4824b6454a26961c6bfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:05.078304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-92e6d5e0-87b0-4d05-a178-2637f4f2dc18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:05.079042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-92e6d5e0-87b0-4d05-a178-2637f4f2dc18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:05.080733 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1010/2047] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:04 2022] DELETE /identity/v3/projects/d012ed4035ac4824b6454a26961c6bfa => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:05.086548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1170575-9327-4014-aeea-06f1ed6d4f64 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:05.087002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1170575-9327-4014-aeea-06f1ed6d4f64 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:05.087088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1170575-9327-4014-aeea-06f1ed6d4f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:05.105071 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b1170575-9327-4014-aeea-06f1ed6d4f64 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:05.106209 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b1170575-9327-4014-aeea-06f1ed6d4f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:05.167481 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1038/2048] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:05.175551 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e6cb2-d1e4-4e37-9cdb-4f13344ea2e0 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:05.175896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e6cb2-d1e4-4e37-9cdb-4f13344ea2e0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:05.176174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e6cb2-d1e4-4e37-9cdb-4f13344ea2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:05.194400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-4b6e6cb2-d1e4-4e37-9cdb-4f13344ea2e0 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:05.195590 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4b6e6cb2-d1e4-4e37-9cdb-4f13344ea2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:05.204105 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1011/2049] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:05.208238 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1170575-9327-4014-aeea-06f1ed6d4f64 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:05.217796 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1170575-9327-4014-aeea-06f1ed6d4f64 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:05.222521 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffca16b-931e-4746-aa72-7d165fe3ba8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:05.223181 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffca16b-931e-4746-aa72-7d165fe3ba8f admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:05.223460 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffca16b-931e-4746-aa72-7d165fe3ba8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:05.223766 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffca16b-931e-4746-aa72-7d165fe3ba8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:05.224528 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffca16b-931e-4746-aa72-7d165fe3ba8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:05.224828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffca16b-931e-4746-aa72-7d165fe3ba8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:05.226426 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffca16b-931e-4746-aa72-7d165fe3ba8f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:05.238034 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1039/2050] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:17:05.242461 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e6cb2-d1e4-4e37-9cdb-4f13344ea2e0 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:05.251994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e6cb2-d1e4-4e37-9cdb-4f13344ea2e0 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:05.257053 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12084165-d791-4626-ad6c-20f9a370a7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:05.257710 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12084165-d791-4626-ad6c-20f9a370a7ba admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:05.257990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12084165-d791-4626-ad6c-20f9a370a7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:05.258265 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12084165-d791-4626-ad6c-20f9a370a7ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:05.258950 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12084165-d791-4626-ad6c-20f9a370a7ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:05.260894 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12084165-d791-4626-ad6c-20f9a370a7ba admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:05.288240 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1012/2051] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:05 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:17:05.292356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffca16b-931e-4746-aa72-7d165fe3ba8f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:05.302231 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffca16b-931e-4746-aa72-7d165fe3ba8f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:05.357837 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19918c2c-702b-4759-bba3-157093ed618e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:05.358436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19918c2c-702b-4759-bba3-157093ed618e admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:05.358764 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19918c2c-702b-4759-bba3-157093ed618e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:05.359045 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19918c2c-702b-4759-bba3-157093ed618e admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:05.359749 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19918c2c-702b-4759-bba3-157093ed618e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:05.361671 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19918c2c-702b-4759-bba3-157093ed618e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:05.382306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1040/2052] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:05 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:05.386264 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12084165-d791-4626-ad6c-20f9a370a7ba admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:05.397213 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12084165-d791-4626-ad6c-20f9a370a7ba admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:05.402289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef92ae3-bed8-46b9-88eb-8b4edbd74ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:05.402954 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef92ae3-bed8-46b9-88eb-8b4edbd74ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:05.403237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef92ae3-bed8-46b9-88eb-8b4edbd74ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:05.403510 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef92ae3-bed8-46b9-88eb-8b4edbd74ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:05.404265 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef92ae3-bed8-46b9-88eb-8b4edbd74ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:05.405874 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef92ae3-bed8-46b9-88eb-8b4edbd74ba2 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:05.416280 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1013/2053] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:05.420283 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19918c2c-702b-4759-bba3-157093ed618e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:05.430303 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19918c2c-702b-4759-bba3-157093ed618e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:05.435363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea19d6f-2595-48cc-b0fe-71dc4c751650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:05.436047 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea19d6f-2595-48cc-b0fe-71dc4c751650 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:05.436327 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea19d6f-2595-48cc-b0fe-71dc4c751650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:05.436602 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea19d6f-2595-48cc-b0fe-71dc4c751650 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:05.437292 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea19d6f-2595-48cc-b0fe-71dc4c751650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:05.438956 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea19d6f-2595-48cc-b0fe-71dc4c751650 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:05.449119 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1041/2054] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:17:05.453261 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef92ae3-bed8-46b9-88eb-8b4edbd74ba2 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:05.463950 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef92ae3-bed8-46b9-88eb-8b4edbd74ba2 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:05.469661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d43fbd-7569-4713-8377-809810006667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:05.470558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d43fbd-7569-4713-8377-809810006667 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:05.470844 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d43fbd-7569-4713-8377-809810006667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:05.471136 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d43fbd-7569-4713-8377-809810006667 admin admin] PATH_INFO: `/v3/projects/d7be01501aa04a6297298f15391ecd4d/users/a99128300cf7482abd8ebb038b4ed0bc/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:05.487068 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d43fbd-7569-4713-8377-809810006667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7be01501aa04a6297298f15391ecd4d, user_id=a99128300cf7482abd8ebb038b4ed0bc, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:05.490193 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d43fbd-7569-4713-8377-809810006667 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:05.506286 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1014/2055] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:05 2022] PUT /identity/v3/projects/d7be01501aa04a6297298f15391ecd4d/users/a99128300cf7482abd8ebb038b4ed0bc/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:05.517799 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b235fe-9f19-4f4d-b504-aa3fed89a539 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:05.518099 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b235fe-9f19-4f4d-b504-aa3fed89a539 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:05.518368 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b235fe-9f19-4f4d-b504-aa3fed89a539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:05.545520 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d3b235fe-9f19-4f4d-b504-aa3fed89a539 None None] MFA Rules not processed for user `a99128300cf7482abd8ebb038b4ed0bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:05.546770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d3b235fe-9f19-4f4d-b504-aa3fed89a539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:05.577051 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:05.611847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1042/2056] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:05.623637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d43fbd-7569-4713-8377-809810006667 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:05.626570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-58d43fbd-7569-4713-8377-809810006667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:05.698559 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d43fbd-7569-4713-8377-809810006667 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:05.705839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb017b93-da3b-4937-ae56-82665cddb06f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b43db1fd0b494a3f8588b498b585f247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:05.706729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb017b93-da3b-4937-ae56-82665cddb06f service glance] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:05.707007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb017b93-da3b-4937-ae56-82665cddb06f service glance] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:05.707281 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb017b93-da3b-4937-ae56-82665cddb06f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:05.719951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb017b93-da3b-4937-ae56-82665cddb06f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:05.721638 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb017b93-da3b-4937-ae56-82665cddb06f service glance] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:05.731069 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1015/2057] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:08.427228 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b235fe-9f19-4f4d-b504-aa3fed89a539 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:08.428772 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d3b235fe-9f19-4f4d-b504-aa3fed89a539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:08.510350 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b235fe-9f19-4f4d-b504-aa3fed89a539 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:08.516350 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa040db-a41d-46d9-837a-9131063c242e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:08.517568 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa040db-a41d-46d9-837a-9131063c242e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:08.517843 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa040db-a41d-46d9-837a-9131063c242e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:08.518112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa040db-a41d-46d9-837a-9131063c242e admin admin] PATH_INFO: `/v3/users/547a6e2319d74129b9a08c7baf49e6c9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:08.519373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa040db-a41d-46d9-837a-9131063c242e admin admin] RBAC: Authorizing `identity:delete_user(user_id=547a6e2319d74129b9a08c7baf49e6c9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:08.521348 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa040db-a41d-46d9-837a-9131063c242e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:08.572086 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-faa040db-a41d-46d9-837a-9131063c242e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '547a6e2319d74129b9a08c7baf49e6c9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:08.572832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-faa040db-a41d-46d9-837a-9131063c242e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '547a6e2319d74129b9a08c7baf49e6c9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:08.573589 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-faa040db-a41d-46d9-837a-9131063c242e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '547a6e2319d74129b9a08c7baf49e6c9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:08.588304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-faa040db-a41d-46d9-837a-9131063c242e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '547a6e2319d74129b9a08c7baf49e6c9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:08.596295 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-faa040db-a41d-46d9-837a-9131063c242e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '547a6e2319d74129b9a08c7baf49e6c9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:08.612539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1043/2058] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:08 2022] DELETE /identity/v3/users/547a6e2319d74129b9a08c7baf49e6c9 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:08.617297 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17beca92-d875-44a6-8ddc-1aeb183166b3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:08.618201 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17beca92-d875-44a6-8ddc-1aeb183166b3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:08.618542 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17beca92-d875-44a6-8ddc-1aeb183166b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:08.638417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-17beca92-d875-44a6-8ddc-1aeb183166b3 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:08.639548 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-17beca92-d875-44a6-8ddc-1aeb183166b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:08.681096 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa040db-a41d-46d9-837a-9131063c242e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:08.682906 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-faa040db-a41d-46d9-837a-9131063c242e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:08.734930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1016/2059] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:08.736000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17beca92-d875-44a6-8ddc-1aeb183166b3 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:08.737701 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-17beca92-d875-44a6-8ddc-1aeb183166b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:08.746957 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:08.768470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa040db-a41d-46d9-837a-9131063c242e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:08.774476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0f98a-8e84-4aa0-b7b5-3f78fe98452d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:08.775478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:08.775865 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0f98a-8e84-4aa0-b7b5-3f78fe98452d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:08.776175 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0f98a-8e84-4aa0-b7b5-3f78fe98452d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:08.776454 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0f98a-8e84-4aa0-b7b5-3f78fe98452d admin admin] PATH_INFO: `/v3/users/67209d464f77471c8ccf9ef3baf95437` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:08.777933 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0f98a-8e84-4aa0-b7b5-3f78fe98452d admin admin] RBAC: Authorizing `identity:delete_user(user_id=67209d464f77471c8ccf9ef3baf95437)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:08.780125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0f98a-8e84-4aa0-b7b5-3f78fe98452d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:08.825488 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17beca92-d875-44a6-8ddc-1aeb183166b3 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:08.829555 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102e0ee-9bf2-4ba3-80dc-9a0f2d3a1273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:08.830254 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102e0ee-9bf2-4ba3-80dc-9a0f2d3a1273 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:08.830533 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102e0ee-9bf2-4ba3-80dc-9a0f2d3a1273 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:08.830968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102e0ee-9bf2-4ba3-80dc-9a0f2d3a1273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:08.841656 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102e0ee-9bf2-4ba3-80dc-9a0f2d3a1273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:08.842111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d5a0f98a-8e84-4aa0-b7b5-3f78fe98452d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67209d464f77471c8ccf9ef3baf95437'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:08.842854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d5a0f98a-8e84-4aa0-b7b5-3f78fe98452d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67209d464f77471c8ccf9ef3baf95437'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:08.843164 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102e0ee-9bf2-4ba3-80dc-9a0f2d3a1273 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:08.843588 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d5a0f98a-8e84-4aa0-b7b5-3f78fe98452d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67209d464f77471c8ccf9ef3baf95437'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:08.856975 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d5a0f98a-8e84-4aa0-b7b5-3f78fe98452d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67209d464f77471c8ccf9ef3baf95437'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:08.860575 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1017/2060] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:08.874240 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d5a0f98a-8e84-4aa0-b7b5-3f78fe98452d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67209d464f77471c8ccf9ef3baf95437'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:08.888238 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1044/2061] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:08 2022] DELETE /identity/v3/users/67209d464f77471c8ccf9ef3baf95437 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:08.893156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88ccd1-82da-44f5-9535-fc726f4a8b49 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:08.893379 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88ccd1-82da-44f5-9535-fc726f4a8b49 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:08.893652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88ccd1-82da-44f5-9535-fc726f4a8b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:08.911444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ec88ccd1-82da-44f5-9535-fc726f4a8b49 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:08.912809 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ec88ccd1-82da-44f5-9535-fc726f4a8b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:09.007211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1018/2062] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:09.017719 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0f98a-8e84-4aa0-b7b5-3f78fe98452d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:09.019181 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d5a0f98a-8e84-4aa0-b7b5-3f78fe98452d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:09.088941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0f98a-8e84-4aa0-b7b5-3f78fe98452d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:09.092919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed0c75-d621-431e-b25e-01f060cef4bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:09.093629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed0c75-d621-431e-b25e-01f060cef4bb service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:09.093902 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed0c75-d621-431e-b25e-01f060cef4bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:09.094238 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed0c75-d621-431e-b25e-01f060cef4bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:09.105103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed0c75-d621-431e-b25e-01f060cef4bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:09.106616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed0c75-d621-431e-b25e-01f060cef4bb service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:09.116606 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1045/2063] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:17:09.157348 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88ccd1-82da-44f5-9535-fc726f4a8b49 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:09.159220 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ec88ccd1-82da-44f5-9535-fc726f4a8b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:09.174793 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88ccd1-82da-44f5-9535-fc726f4a8b49 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:09.185039 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c672f02-be54-4b64-9f59-0d593454aaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:09.187316 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c672f02-be54-4b64-9f59-0d593454aaef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:09.187757 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c672f02-be54-4b64-9f59-0d593454aaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:09.188196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c672f02-be54-4b64-9f59-0d593454aaef admin admin] PATH_INFO: `/v3/projects/910d9eb138824079aa6acd95ab6f0e06` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:09.190190 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c672f02-be54-4b64-9f59-0d593454aaef admin admin] RBAC: Authorizing `identity:delete_project(project_id=910d9eb138824079aa6acd95ab6f0e06)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:09.192478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c672f02-be54-4b64-9f59-0d593454aaef admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:09.246733 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9c672f02-be54-4b64-9f59-0d593454aaef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '910d9eb138824079aa6acd95ab6f0e06', 'request_id': 'req-9c672f02-be54-4b64-9f59-0d593454aaef'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:09.257215 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9c672f02-be54-4b64-9f59-0d593454aaef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '910d9eb138824079aa6acd95ab6f0e06', 'request_id': 'req-9c672f02-be54-4b64-9f59-0d593454aaef'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:09.264278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9c672f02-be54-4b64-9f59-0d593454aaef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '910d9eb138824079aa6acd95ab6f0e06', 'request_id': 'req-9c672f02-be54-4b64-9f59-0d593454aaef'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:09.282380 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9c672f02-be54-4b64-9f59-0d593454aaef admin admin] The token cache is being invalidate because project 910d9eb138824079aa6acd95ab6f0e06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:09.282916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9c672f02-be54-4b64-9f59-0d593454aaef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:09.283927 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9c672f02-be54-4b64-9f59-0d593454aaef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:09.286127 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1019/2064] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:09 2022] DELETE /identity/v3/projects/910d9eb138824079aa6acd95ab6f0e06 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:09.289857 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed0c75-d621-431e-b25e-01f060cef4bb service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:09.301952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed0c75-d621-431e-b25e-01f060cef4bb service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:09.373669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139c200-46bd-4dc0-9490-24fc5a8c26cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:09.374658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139c200-46bd-4dc0-9490-24fc5a8c26cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:09.375007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139c200-46bd-4dc0-9490-24fc5a8c26cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:09.375365 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139c200-46bd-4dc0-9490-24fc5a8c26cd admin admin] PATH_INFO: `/v3/users/a1f01fc03b6449faa41c76aa03ed281e` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:09.376698 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139c200-46bd-4dc0-9490-24fc5a8c26cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1f01fc03b6449faa41c76aa03ed281e)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:09.379177 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139c200-46bd-4dc0-9490-24fc5a8c26cd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:09.394223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:09.431522 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c672f02-be54-4b64-9f59-0d593454aaef admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:09.433298 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9c672f02-be54-4b64-9f59-0d593454aaef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:09.436964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0139c200-46bd-4dc0-9490-24fc5a8c26cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1f01fc03b6449faa41c76aa03ed281e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:09.437965 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0139c200-46bd-4dc0-9490-24fc5a8c26cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1f01fc03b6449faa41c76aa03ed281e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:09.438877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0139c200-46bd-4dc0-9490-24fc5a8c26cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1f01fc03b6449faa41c76aa03ed281e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:09.460769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c672f02-be54-4b64-9f59-0d593454aaef admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:09.461087 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0139c200-46bd-4dc0-9490-24fc5a8c26cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1f01fc03b6449faa41c76aa03ed281e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:09.473124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0139c200-46bd-4dc0-9490-24fc5a8c26cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1f01fc03b6449faa41c76aa03ed281e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:09.474365 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac7759-466b-4bc2-9367-6e17f38816e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:09.476209 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac7759-466b-4bc2-9367-6e17f38816e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:09.478170 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac7759-466b-4bc2-9367-6e17f38816e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:09.478756 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac7759-466b-4bc2-9367-6e17f38816e5 admin admin] PATH_INFO: `/v3/projects/e92d327de86441be99184405591568f7` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:09.481853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac7759-466b-4bc2-9367-6e17f38816e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e92d327de86441be99184405591568f7)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:09.487659 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac7759-466b-4bc2-9367-6e17f38816e5 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:09.498335 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1046/2065] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:09 2022] DELETE /identity/v3/users/a1f01fc03b6449faa41c76aa03ed281e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:09.564780 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:09.568630 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e5ac7759-466b-4bc2-9367-6e17f38816e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e92d327de86441be99184405591568f7', 'request_id': 'req-e5ac7759-466b-4bc2-9367-6e17f38816e5'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:09.585233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e5ac7759-466b-4bc2-9367-6e17f38816e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e92d327de86441be99184405591568f7', 'request_id': 'req-e5ac7759-466b-4bc2-9367-6e17f38816e5'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:09.596594 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e5ac7759-466b-4bc2-9367-6e17f38816e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e92d327de86441be99184405591568f7', 'request_id': 'req-e5ac7759-466b-4bc2-9367-6e17f38816e5'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:09.618614 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e5ac7759-466b-4bc2-9367-6e17f38816e5 admin admin] The token cache is being invalidate because project e92d327de86441be99184405591568f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:09.619174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e5ac7759-466b-4bc2-9367-6e17f38816e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:09.620015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e5ac7759-466b-4bc2-9367-6e17f38816e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:09.622442 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1020/2066] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:09 2022] DELETE /identity/v3/projects/e92d327de86441be99184405591568f7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:17:09.627862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ba856-ea15-4309-9941-474cf29889c4 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:09.628252 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ba856-ea15-4309-9941-474cf29889c4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:09.628739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ba856-ea15-4309-9941-474cf29889c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:09.659178 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2d8ba856-ea15-4309-9941-474cf29889c4 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:09.660516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2d8ba856-ea15-4309-9941-474cf29889c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:09.739391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1047/2067] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:09.747711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaf50f-9405-4a45-9a86-9fc70d4f9dbe None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:09.748004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaf50f-9405-4a45-9a86-9fc70d4f9dbe None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:09.748280 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaf50f-9405-4a45-9a86-9fc70d4f9dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:09.778720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-fdfaf50f-9405-4a45-9a86-9fc70d4f9dbe None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:09.793660 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fdfaf50f-9405-4a45-9a86-9fc70d4f9dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:09.809671 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1021/2068] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:09.813814 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ba856-ea15-4309-9941-474cf29889c4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:09.833199 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ba856-ea15-4309-9941-474cf29889c4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:09.839501 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28f439-d0c4-430c-9a6c-dded31d7b124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:09.840230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28f439-d0c4-430c-9a6c-dded31d7b124 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:09.840560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28f439-d0c4-430c-9a6c-dded31d7b124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:09.840838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28f439-d0c4-430c-9a6c-dded31d7b124 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:09.842368 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28f439-d0c4-430c-9a6c-dded31d7b124 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:09.842551 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28f439-d0c4-430c-9a6c-dded31d7b124 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:09.844332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28f439-d0c4-430c-9a6c-dded31d7b124 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:09.860919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1048/2069] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:17:09.865430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfaf50f-9405-4a45-9a86-9fc70d4f9dbe None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:09.875415 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfaf50f-9405-4a45-9a86-9fc70d4f9dbe None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:09.880388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b85fb-6235-4111-a2c0-8cd76a3eeb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:09.881069 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b85fb-6235-4111-a2c0-8cd76a3eeb6e admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:09.881365 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b85fb-6235-4111-a2c0-8cd76a3eeb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:09.881621 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b85fb-6235-4111-a2c0-8cd76a3eeb6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:09.882384 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b85fb-6235-4111-a2c0-8cd76a3eeb6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:09.884461 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b85fb-6235-4111-a2c0-8cd76a3eeb6e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:09.916684 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28f439-d0c4-430c-9a6c-dded31d7b124 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:09.918920 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0c28f439-d0c4-430c-9a6c-dded31d7b124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:09.930579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1022/2070] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:09 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:09.935590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28f439-d0c4-430c-9a6c-dded31d7b124 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:09.942143 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b85fb-6235-4111-a2c0-8cd76a3eeb6e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:10.011484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c7f83-6b33-4341-9f00-3041bbda4d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:10.012445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7f83-6b33-4341-9f00-3041bbda4d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:10.012917 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7f83-6b33-4341-9f00-3041bbda4d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:10.013560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7f83-6b33-4341-9f00-3041bbda4d2a admin admin] PATH_INFO: `/v3/projects/66476f9b51e54d6c8e2f68c30b09c0bc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:10.014613 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c7f83-6b33-4341-9f00-3041bbda4d2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=66476f9b51e54d6c8e2f68c30b09c0bc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:10.016624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c7f83-6b33-4341-9f00-3041bbda4d2a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:10.025500 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b85fb-6235-4111-a2c0-8cd76a3eeb6e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:10.031471 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247bf545-c1cb-4892-9523-85cb4c9aa90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:10.032206 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247bf545-c1cb-4892-9523-85cb4c9aa90b admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:10.032479 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247bf545-c1cb-4892-9523-85cb4c9aa90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:10.032763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247bf545-c1cb-4892-9523-85cb4c9aa90b admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:10.033606 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247bf545-c1cb-4892-9523-85cb4c9aa90b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:10.038712 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247bf545-c1cb-4892-9523-85cb4c9aa90b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:10.044589 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:10.062444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1023/2071] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:09 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:17:10.066607 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247bf545-c1cb-4892-9523-85cb4c9aa90b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:10.074320 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4c9c7f83-6b33-4341-9f00-3041bbda4d2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66476f9b51e54d6c8e2f68c30b09c0bc', 'request_id': 'req-4c9c7f83-6b33-4341-9f00-3041bbda4d2a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:10.076278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247bf545-c1cb-4892-9523-85cb4c9aa90b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:10.082656 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4c9c7f83-6b33-4341-9f00-3041bbda4d2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66476f9b51e54d6c8e2f68c30b09c0bc', 'request_id': 'req-4c9c7f83-6b33-4341-9f00-3041bbda4d2a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:10.088536 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4c9c7f83-6b33-4341-9f00-3041bbda4d2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66476f9b51e54d6c8e2f68c30b09c0bc', 'request_id': 'req-4c9c7f83-6b33-4341-9f00-3041bbda4d2a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:10.100086 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4c9c7f83-6b33-4341-9f00-3041bbda4d2a admin admin] The token cache is being invalidate because project 66476f9b51e54d6c8e2f68c30b09c0bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:10.100370 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4c9c7f83-6b33-4341-9f00-3041bbda4d2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:10.101040 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4c9c7f83-6b33-4341-9f00-3041bbda4d2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:10.102601 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1049/2072] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:09 2022] DELETE /identity/v3/projects/66476f9b51e54d6c8e2f68c30b09c0bc => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:10.106602 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c7f83-6b33-4341-9f00-3041bbda4d2a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:10.107991 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4c9c7f83-6b33-4341-9f00-3041bbda4d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:10.139538 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d83ac-a732-4e84-80a4-86efe0ce361c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:10.140283 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d83ac-a732-4e84-80a4-86efe0ce361c admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:10.140565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d83ac-a732-4e84-80a4-86efe0ce361c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:10.140840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d83ac-a732-4e84-80a4-86efe0ce361c admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:10.141606 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d83ac-a732-4e84-80a4-86efe0ce361c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:10.143522 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d83ac-a732-4e84-80a4-86efe0ce361c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:10.154249 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1024/2073] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:10.158693 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d83ac-a732-4e84-80a4-86efe0ce361c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:10.160477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8d8d83ac-a732-4e84-80a4-86efe0ce361c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:10.173504 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d83ac-a732-4e84-80a4-86efe0ce361c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:10.177899 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c7f83-6b33-4341-9f00-3041bbda4d2a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:10.178072 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a925d-b029-4100-a7ac-dac33e07a73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:10.178725 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a925d-b029-4100-a7ac-dac33e07a73f admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:10.179138 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a925d-b029-4100-a7ac-dac33e07a73f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:10.179414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a925d-b029-4100-a7ac-dac33e07a73f admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:10.180182 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a925d-b029-4100-a7ac-dac33e07a73f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:10.181774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a925d-b029-4100-a7ac-dac33e07a73f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:10.182594 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69745e0b-1b04-4bfa-b7d1-7f8292258c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:10.183477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69745e0b-1b04-4bfa-b7d1-7f8292258c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:10.183793 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69745e0b-1b04-4bfa-b7d1-7f8292258c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:10.184172 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69745e0b-1b04-4bfa-b7d1-7f8292258c8e admin admin] PATH_INFO: `/v3/users/59277683010b460f972ccb3592314277` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:10.185378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69745e0b-1b04-4bfa-b7d1-7f8292258c8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=59277683010b460f972ccb3592314277)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:10.187314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69745e0b-1b04-4bfa-b7d1-7f8292258c8e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:10.191798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1025/2074] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:17:10.195794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a925d-b029-4100-a7ac-dac33e07a73f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:10.205869 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a925d-b029-4100-a7ac-dac33e07a73f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:10.210665 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79feef18-f578-4b89-ae57-0d135f089050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:10.211538 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79feef18-f578-4b89-ae57-0d135f089050 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:10.211855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79feef18-f578-4b89-ae57-0d135f089050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:10.212125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79feef18-f578-4b89-ae57-0d135f089050 admin admin] PATH_INFO: `/v3/projects/a82566e398014a6a9c20d19dc3a10682/users/16987823bee5458096941bdbe8534a63/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:10.227836 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79feef18-f578-4b89-ae57-0d135f089050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a82566e398014a6a9c20d19dc3a10682, user_id=16987823bee5458096941bdbe8534a63, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:10.230568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79feef18-f578-4b89-ae57-0d135f089050 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:10.233816 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=4,keystone:SELECT=90,keystone:INSERT=6 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:10.236061 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-69745e0b-1b04-4bfa-b7d1-7f8292258c8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59277683010b460f972ccb3592314277'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:10.236798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-69745e0b-1b04-4bfa-b7d1-7f8292258c8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59277683010b460f972ccb3592314277'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:10.237746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-69745e0b-1b04-4bfa-b7d1-7f8292258c8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59277683010b460f972ccb3592314277'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:10.246181 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1026/2075] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:10 2022] PUT /identity/v3/projects/a82566e398014a6a9c20d19dc3a10682/users/16987823bee5458096941bdbe8534a63/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:10.250677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-69745e0b-1b04-4bfa-b7d1-7f8292258c8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59277683010b460f972ccb3592314277'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:10.256341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445232b3-98ea-4a6b-8334-a8fd1d7cdf49 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:10.256628 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445232b3-98ea-4a6b-8334-a8fd1d7cdf49 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:10.256905 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445232b3-98ea-4a6b-8334-a8fd1d7cdf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:10.257905 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-69745e0b-1b04-4bfa-b7d1-7f8292258c8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59277683010b460f972ccb3592314277'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:10.270909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1050/2076] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:10 2022] DELETE /identity/v3/users/59277683010b460f972ccb3592314277 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:17:10.285998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-445232b3-98ea-4a6b-8334-a8fd1d7cdf49 None None] MFA Rules not processed for user `16987823bee5458096941bdbe8534a63`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:10.287274 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-445232b3-98ea-4a6b-8334-a8fd1d7cdf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:10.358458 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1027/2077] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:10.370666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69745e0b-1b04-4bfa-b7d1-7f8292258c8e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:10.372901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-69745e0b-1b04-4bfa-b7d1-7f8292258c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:10.419024 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:10.461259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69745e0b-1b04-4bfa-b7d1-7f8292258c8e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:10.465897 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e8800-56e9-47ab-96d1-95c4c396a173 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:10.466561 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e8800-56e9-47ab-96d1-95c4c396a173 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:10.466847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e8800-56e9-47ab-96d1-95c4c396a173 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:10.467249 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e8800-56e9-47ab-96d1-95c4c396a173 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:10.478947 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e8800-56e9-47ab-96d1-95c4c396a173 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:10.480641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e8800-56e9-47ab-96d1-95c4c396a173 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:10.493169 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1051/2078] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:10.569498 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445232b3-98ea-4a6b-8334-a8fd1d7cdf49 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:10.571240 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-445232b3-98ea-4a6b-8334-a8fd1d7cdf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:10.645845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445232b3-98ea-4a6b-8334-a8fd1d7cdf49 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:10.650689 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be7792-8ec5-4661-b6a5-6c7957661725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:10.651680 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be7792-8ec5-4661-b6a5-6c7957661725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:10.652017 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be7792-8ec5-4661-b6a5-6c7957661725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:10.652298 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be7792-8ec5-4661-b6a5-6c7957661725 admin admin] PATH_INFO: `/v3/projects/eacde9cac83647bfbb71197d4bf3ba79` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:10.653789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be7792-8ec5-4661-b6a5-6c7957661725 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eacde9cac83647bfbb71197d4bf3ba79)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:10.656143 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be7792-8ec5-4661-b6a5-6c7957661725 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:10.706969 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-70be7792-8ec5-4661-b6a5-6c7957661725 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eacde9cac83647bfbb71197d4bf3ba79', 'request_id': 'req-70be7792-8ec5-4661-b6a5-6c7957661725'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:10.719033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-70be7792-8ec5-4661-b6a5-6c7957661725 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eacde9cac83647bfbb71197d4bf3ba79', 'request_id': 'req-70be7792-8ec5-4661-b6a5-6c7957661725'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:10.725361 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-70be7792-8ec5-4661-b6a5-6c7957661725 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eacde9cac83647bfbb71197d4bf3ba79', 'request_id': 'req-70be7792-8ec5-4661-b6a5-6c7957661725'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:10.737114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-70be7792-8ec5-4661-b6a5-6c7957661725 admin admin] The token cache is being invalidate because project eacde9cac83647bfbb71197d4bf3ba79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:10.737405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-70be7792-8ec5-4661-b6a5-6c7957661725 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:10.738051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-70be7792-8ec5-4661-b6a5-6c7957661725 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:10.739991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1028/2079] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:10 2022] DELETE /identity/v3/projects/eacde9cac83647bfbb71197d4bf3ba79 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:10.745394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3d60a-b2c4-4d18-acf2-4591188f26ac None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:10.745732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3d60a-b2c4-4d18-acf2-4591188f26ac None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:10.746137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3d60a-b2c4-4d18-acf2-4591188f26ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:10.768009 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9cc3d60a-b2c4-4d18-acf2-4591188f26ac None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:10.769224 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9cc3d60a-b2c4-4d18-acf2-4591188f26ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:10.837839 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1052/2080] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:10.846856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5094dfc-fe6c-4efe-94a5-5b89db030fde None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:10.847144 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5094dfc-fe6c-4efe-94a5-5b89db030fde None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:10.847467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5094dfc-fe6c-4efe-94a5-5b89db030fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:10.866045 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d5094dfc-fe6c-4efe-94a5-5b89db030fde None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:10.867250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d5094dfc-fe6c-4efe-94a5-5b89db030fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:10.875910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1029/2081] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:10.881865 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3d60a-b2c4-4d18-acf2-4591188f26ac None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:10.898605 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3d60a-b2c4-4d18-acf2-4591188f26ac None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:10.905998 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe5bf7-458e-4f5e-91eb-6e89005a7249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:10.906668 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe5bf7-458e-4f5e-91eb-6e89005a7249 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:10.906955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe5bf7-458e-4f5e-91eb-6e89005a7249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:10.907232 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe5bf7-458e-4f5e-91eb-6e89005a7249 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:10.908067 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe5bf7-458e-4f5e-91eb-6e89005a7249 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:10.908529 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe5bf7-458e-4f5e-91eb-6e89005a7249 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:10.910504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe5bf7-458e-4f5e-91eb-6e89005a7249 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:10.925800 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1053/2082] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:17:10.930152 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5094dfc-fe6c-4efe-94a5-5b89db030fde None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:10.940022 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5094dfc-fe6c-4efe-94a5-5b89db030fde None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:10.945000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0c99c-ff3d-4376-8bec-7ba0c462641a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:10.945654 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0c99c-ff3d-4376-8bec-7ba0c462641a admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:10.945912 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0c99c-ff3d-4376-8bec-7ba0c462641a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:10.946205 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0c99c-ff3d-4376-8bec-7ba0c462641a admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:10.946888 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0c99c-ff3d-4376-8bec-7ba0c462641a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:10.948846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0c99c-ff3d-4376-8bec-7ba0c462641a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:10.953260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:10.985631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1030/2083] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:10 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:10.994923 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe5bf7-458e-4f5e-91eb-6e89005a7249 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:11.007515 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe5bf7-458e-4f5e-91eb-6e89005a7249 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:11.087299 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0462f-4b55-4560-9dd8-359490861312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:11.088107 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0462f-4b55-4560-9dd8-359490861312 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:11.088368 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0462f-4b55-4560-9dd8-359490861312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:11.088663 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0462f-4b55-4560-9dd8-359490861312 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:11.089358 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0462f-4b55-4560-9dd8-359490861312 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:11.091404 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0462f-4b55-4560-9dd8-359490861312 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:11.114217 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1054/2084] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:10 2022] POST /identity/v3/users => generated 283 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:11.118611 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0c99c-ff3d-4376-8bec-7ba0c462641a admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:11.131625 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0c99c-ff3d-4376-8bec-7ba0c462641a admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:11.140237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0dae7-636c-4785-ba6b-e1fd0dedeafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:11.140990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0dae7-636c-4785-ba6b-e1fd0dedeafa admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:11.141279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0dae7-636c-4785-ba6b-e1fd0dedeafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:11.141560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0dae7-636c-4785-ba6b-e1fd0dedeafa admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:11.142672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0dae7-636c-4785-ba6b-e1fd0dedeafa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:11.146060 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0dae7-636c-4785-ba6b-e1fd0dedeafa admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:11.158709 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1031/2085] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:11.163359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0462f-4b55-4560-9dd8-359490861312 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:11.176463 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0462f-4b55-4560-9dd8-359490861312 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:11.181445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5dfbfe-26ca-42b4-87a0-e9bae2fd84c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:11.182366 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dfbfe-26ca-42b4-87a0-e9bae2fd84c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:11.182824 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dfbfe-26ca-42b4-87a0-e9bae2fd84c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:11.183269 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dfbfe-26ca-42b4-87a0-e9bae2fd84c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:11.184241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5dfbfe-26ca-42b4-87a0-e9bae2fd84c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:11.186051 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5dfbfe-26ca-42b4-87a0-e9bae2fd84c0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:11.197583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1055/2086] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:11.201851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0dae7-636c-4785-ba6b-e1fd0dedeafa admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:11.214483 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0dae7-636c-4785-ba6b-e1fd0dedeafa admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:11.220491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e40385-260e-4c7d-9de9-c583b80c82f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:11.221630 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e40385-260e-4c7d-9de9-c583b80c82f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:11.222093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e40385-260e-4c7d-9de9-c583b80c82f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:11.222699 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e40385-260e-4c7d-9de9-c583b80c82f2 admin admin] PATH_INFO: `/v3/projects/cf54b3443056486f9575a111d25d928e/users/91fcd1d4ce2c49e6a67121e9c5398a35/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:11.255720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e40385-260e-4c7d-9de9-c583b80c82f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf54b3443056486f9575a111d25d928e, user_id=91fcd1d4ce2c49e6a67121e9c5398a35, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:11.258536 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e40385-260e-4c7d-9de9-c583b80c82f2 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:11.277460 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1032/2087] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:11 2022] PUT /identity/v3/projects/cf54b3443056486f9575a111d25d928e/users/91fcd1d4ce2c49e6a67121e9c5398a35/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:11.288048 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931943ea-e6cd-47b7-a5ec-b089eb580c59 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:11.288575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931943ea-e6cd-47b7-a5ec-b089eb580c59 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:11.288727 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931943ea-e6cd-47b7-a5ec-b089eb580c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:11.317602 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-931943ea-e6cd-47b7-a5ec-b089eb580c59 None None] MFA Rules not processed for user `91fcd1d4ce2c49e6a67121e9c5398a35`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:11.319093 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-931943ea-e6cd-47b7-a5ec-b089eb580c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:11.348714 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:11.390404 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1056/2088] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:11 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:11.402465 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e40385-260e-4c7d-9de9-c583b80c82f2 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:11.404651 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c1e40385-260e-4c7d-9de9-c583b80c82f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:11.473011 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e40385-260e-4c7d-9de9-c583b80c82f2 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:11.476944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4e4b3-9706-492c-a134-df32b66a7183 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:11.477602 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4e4b3-9706-492c-a134-df32b66a7183 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:11.477883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4e4b3-9706-492c-a134-df32b66a7183 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:11.478155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4e4b3-9706-492c-a134-df32b66a7183 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:11.486675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4e4b3-9706-492c-a134-df32b66a7183 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:11.488167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4e4b3-9706-492c-a134-df32b66a7183 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:11.497581 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1033/2089] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:11 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:13.070678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931943ea-e6cd-47b7-a5ec-b089eb580c59 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:13.073960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-931943ea-e6cd-47b7-a5ec-b089eb580c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:13.168196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931943ea-e6cd-47b7-a5ec-b089eb580c59 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:13.176810 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f3881-308a-4398-8112-95a77b4602cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:13.178280 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f3881-308a-4398-8112-95a77b4602cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:13.178940 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f3881-308a-4398-8112-95a77b4602cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:13.179371 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f3881-308a-4398-8112-95a77b4602cc admin admin] PATH_INFO: `/v3/users/16987823bee5458096941bdbe8534a63` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:13.181079 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f3881-308a-4398-8112-95a77b4602cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=16987823bee5458096941bdbe8534a63)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:13.183202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f3881-308a-4398-8112-95a77b4602cc admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:13.232102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-489f3881-308a-4398-8112-95a77b4602cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16987823bee5458096941bdbe8534a63'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:13.232700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-489f3881-308a-4398-8112-95a77b4602cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16987823bee5458096941bdbe8534a63'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:13.233300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-489f3881-308a-4398-8112-95a77b4602cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16987823bee5458096941bdbe8534a63'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:13.245343 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-489f3881-308a-4398-8112-95a77b4602cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16987823bee5458096941bdbe8534a63'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:13.252460 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-489f3881-308a-4398-8112-95a77b4602cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16987823bee5458096941bdbe8534a63'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:13.270741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1057/2090] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:13 2022] DELETE /identity/v3/users/16987823bee5458096941bdbe8534a63 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:13.275739 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91aaa3-e6a4-4903-a70f-68eabe0ed3b0 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:13.276202 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91aaa3-e6a4-4903-a70f-68eabe0ed3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:13.276562 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91aaa3-e6a4-4903-a70f-68eabe0ed3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:13.294750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1b91aaa3-e6a4-4903-a70f-68eabe0ed3b0 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:13.296061 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1b91aaa3-e6a4-4903-a70f-68eabe0ed3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:13.360426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1034/2091] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:13.371402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f3881-308a-4398-8112-95a77b4602cc admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:13.372972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-489f3881-308a-4398-8112-95a77b4602cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:13.430496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:13.443680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f3881-308a-4398-8112-95a77b4602cc admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:13.449289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a683fe0-9435-4c4a-b765-75806bb4d9f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:13.450313 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a683fe0-9435-4c4a-b765-75806bb4d9f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:13.450555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a683fe0-9435-4c4a-b765-75806bb4d9f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:13.450854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a683fe0-9435-4c4a-b765-75806bb4d9f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:13.463185 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a683fe0-9435-4c4a-b765-75806bb4d9f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:13.464684 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a683fe0-9435-4c4a-b765-75806bb4d9f1 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:13.475649 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1058/2092] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:13.815326 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91aaa3-e6a4-4903-a70f-68eabe0ed3b0 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:13.816937 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1b91aaa3-e6a4-4903-a70f-68eabe0ed3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:13.833473 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91aaa3-e6a4-4903-a70f-68eabe0ed3b0 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:13.850937 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0df5e9-02ea-40b6-b720-525b99ad8b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:13.850937 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0df5e9-02ea-40b6-b720-525b99ad8b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:13.850937 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0df5e9-02ea-40b6-b720-525b99ad8b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:13.850937 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0df5e9-02ea-40b6-b720-525b99ad8b07 admin admin] PATH_INFO: `/v3/projects/a82566e398014a6a9c20d19dc3a10682` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:13.851970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0df5e9-02ea-40b6-b720-525b99ad8b07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a82566e398014a6a9c20d19dc3a10682)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:13.854292 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0df5e9-02ea-40b6-b720-525b99ad8b07 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:13.927059 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7c0df5e9-02ea-40b6-b720-525b99ad8b07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a82566e398014a6a9c20d19dc3a10682', 'request_id': 'req-7c0df5e9-02ea-40b6-b720-525b99ad8b07'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:13.929347 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:13.940422 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7c0df5e9-02ea-40b6-b720-525b99ad8b07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a82566e398014a6a9c20d19dc3a10682', 'request_id': 'req-7c0df5e9-02ea-40b6-b720-525b99ad8b07'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:13.949892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7c0df5e9-02ea-40b6-b720-525b99ad8b07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a82566e398014a6a9c20d19dc3a10682', 'request_id': 'req-7c0df5e9-02ea-40b6-b720-525b99ad8b07'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:13.965802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7c0df5e9-02ea-40b6-b720-525b99ad8b07 admin admin] The token cache is being invalidate because project a82566e398014a6a9c20d19dc3a10682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:13.965802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7c0df5e9-02ea-40b6-b720-525b99ad8b07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:13.966218 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7c0df5e9-02ea-40b6-b720-525b99ad8b07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:13.968004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1035/2093] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:13 2022] DELETE /identity/v3/projects/a82566e398014a6a9c20d19dc3a10682 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:21.661485 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7c4bd-c588-4081-a2a4-6c5c8f113b11 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:21.662308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7c4bd-c588-4081-a2a4-6c5c8f113b11 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:21.662592 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7c4bd-c588-4081-a2a4-6c5c8f113b11 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:21.664520 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1059/2094] 10.222.0.27 () {54 vars in 921 bytes} [Mon Jun 27 21:17:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:17:21.675541 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4ae72-7df0-435c-8265-d2b7833683c7 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:21.675541 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4ae72-7df0-435c-8265-d2b7833683c7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:21.675541 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4ae72-7df0-435c-8265-d2b7833683c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:21.702821 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-57b4ae72-7df0-435c-8265-d2b7833683c7 None None] MFA Rules not processed for user `1f5fa9b0b0314ee9bb7e16b8464307b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:21.704452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-57b4ae72-7df0-435c-8265-d2b7833683c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:21.767421 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7c4bd-c588-4081-a2a4-6c5c8f113b11 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:21.769460 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-76d7c4bd-c588-4081-a2a4-6c5c8f113b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:21.823534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1036/2095] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Jun 27 21:17:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:21.838270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4ae72-7df0-435c-8265-d2b7833683c7 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:21.839826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-57b4ae72-7df0-435c-8265-d2b7833683c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:21.855306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7c4bd-c588-4081-a2a4-6c5c8f113b11 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:21.858602 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4ae72-7df0-435c-8265-d2b7833683c7 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:21.863971 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852557ce-d38d-460d-973c-2bd2c2386dfe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5fa9b0b0314ee9bb7e16b8464307b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:21.864684 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852557ce-d38d-460d-973c-2bd2c2386dfe service placement] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:21.865043 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852557ce-d38d-460d-973c-2bd2c2386dfe service placement] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:21.865381 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852557ce-d38d-460d-973c-2bd2c2386dfe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:21.869977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cd9c9-8c7b-41b3-acc8-c6aa3c54e480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:21.872095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cd9c9-8c7b-41b3-acc8-c6aa3c54e480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:21.873027 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cd9c9-8c7b-41b3-acc8-c6aa3c54e480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:21.873323 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cd9c9-8c7b-41b3-acc8-c6aa3c54e480 admin admin] PATH_INFO: `/v3/users/14307670704943768a05d7d6e5a7318b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:21.875040 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562cd9c9-8c7b-41b3-acc8-c6aa3c54e480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14307670704943768a05d7d6e5a7318b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:21.875830 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852557ce-d38d-460d-973c-2bd2c2386dfe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:21.877776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562cd9c9-8c7b-41b3-acc8-c6aa3c54e480 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:21.878347 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852557ce-d38d-460d-973c-2bd2c2386dfe service placement] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:21.889588 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1037/2096] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:21.947687 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-562cd9c9-8c7b-41b3-acc8-c6aa3c54e480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14307670704943768a05d7d6e5a7318b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:21.948351 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-562cd9c9-8c7b-41b3-acc8-c6aa3c54e480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14307670704943768a05d7d6e5a7318b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:21.948966 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-562cd9c9-8c7b-41b3-acc8-c6aa3c54e480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14307670704943768a05d7d6e5a7318b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:21.960780 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-562cd9c9-8c7b-41b3-acc8-c6aa3c54e480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14307670704943768a05d7d6e5a7318b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:21.967501 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-562cd9c9-8c7b-41b3-acc8-c6aa3c54e480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14307670704943768a05d7d6e5a7318b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:21.983058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1060/2097] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:21 2022] DELETE /identity/v3/users/14307670704943768a05d7d6e5a7318b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:22.164601 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852557ce-d38d-460d-973c-2bd2c2386dfe service placement] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:22.190747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852557ce-d38d-460d-973c-2bd2c2386dfe service placement] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:22.262221 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd45dd-4dcd-4e7f-9ae1-46a1c5c29a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:22.263912 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd45dd-4dcd-4e7f-9ae1-46a1c5c29a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:22.264272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd45dd-4dcd-4e7f-9ae1-46a1c5c29a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:22.265500 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd45dd-4dcd-4e7f-9ae1-46a1c5c29a6c admin admin] PATH_INFO: `/v3/projects/0d49eb39209b4612aeb8b13eff382b84` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:22.267454 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd45dd-4dcd-4e7f-9ae1-46a1c5c29a6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d49eb39209b4612aeb8b13eff382b84)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:22.269922 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd45dd-4dcd-4e7f-9ae1-46a1c5c29a6c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:22.325726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-84cd45dd-4dcd-4e7f-9ae1-46a1c5c29a6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d49eb39209b4612aeb8b13eff382b84', 'request_id': 'req-84cd45dd-4dcd-4e7f-9ae1-46a1c5c29a6c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:22.334593 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-84cd45dd-4dcd-4e7f-9ae1-46a1c5c29a6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d49eb39209b4612aeb8b13eff382b84', 'request_id': 'req-84cd45dd-4dcd-4e7f-9ae1-46a1c5c29a6c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:22.341137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:22.343427 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-84cd45dd-4dcd-4e7f-9ae1-46a1c5c29a6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d49eb39209b4612aeb8b13eff382b84', 'request_id': 'req-84cd45dd-4dcd-4e7f-9ae1-46a1c5c29a6c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:22.358280 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-84cd45dd-4dcd-4e7f-9ae1-46a1c5c29a6c admin admin] The token cache is being invalidate because project 0d49eb39209b4612aeb8b13eff382b84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:22.358640 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-84cd45dd-4dcd-4e7f-9ae1-46a1c5c29a6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:22.367149 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-84cd45dd-4dcd-4e7f-9ae1-46a1c5c29a6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:22.367149 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1038/2098] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:22 2022] DELETE /identity/v3/projects/0d49eb39209b4612aeb8b13eff382b84 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:22.369246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745a18d-c414-461c-a055-03b12b1c3b53 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:22.369568 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745a18d-c414-461c-a055-03b12b1c3b53 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:22.369850 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745a18d-c414-461c-a055-03b12b1c3b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:22.388999 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2745a18d-c414-461c-a055-03b12b1c3b53 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:22.390407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2745a18d-c414-461c-a055-03b12b1c3b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:22.452005 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1061/2099] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:22.459936 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f966163-c3df-4ea6-8da5-85539d18eecb None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:22.460224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f966163-c3df-4ea6-8da5-85539d18eecb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:22.460500 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f966163-c3df-4ea6-8da5-85539d18eecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:22.479382 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-7f966163-c3df-4ea6-8da5-85539d18eecb None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:22.480757 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7f966163-c3df-4ea6-8da5-85539d18eecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:22.490966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1039/2100] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:22.495358 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745a18d-c414-461c-a055-03b12b1c3b53 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:22.505552 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745a18d-c414-461c-a055-03b12b1c3b53 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:22.510061 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75f36e-7ae3-4254-974f-f9cbd00b11af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:22.510734 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75f36e-7ae3-4254-974f-f9cbd00b11af admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:22.511021 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75f36e-7ae3-4254-974f-f9cbd00b11af admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:22.511295 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75f36e-7ae3-4254-974f-f9cbd00b11af admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:22.512173 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75f36e-7ae3-4254-974f-f9cbd00b11af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:22.512488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75f36e-7ae3-4254-974f-f9cbd00b11af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:22.514116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75f36e-7ae3-4254-974f-f9cbd00b11af admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:22.516957 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:22.527533 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1062/2101] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:17:22.534312 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f966163-c3df-4ea6-8da5-85539d18eecb None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:22.544804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f966163-c3df-4ea6-8da5-85539d18eecb None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:22.550005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c76ec3-88cc-4d49-9f4f-29e2195c23a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:22.550683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c76ec3-88cc-4d49-9f4f-29e2195c23a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:22.550966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c76ec3-88cc-4d49-9f4f-29e2195c23a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:22.551243 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c76ec3-88cc-4d49-9f4f-29e2195c23a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:22.551982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c76ec3-88cc-4d49-9f4f-29e2195c23a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:22.553915 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c76ec3-88cc-4d49-9f4f-29e2195c23a7 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:22.580889 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1040/2102] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:22 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:22.584691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75f36e-7ae3-4254-974f-f9cbd00b11af admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:22.594970 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75f36e-7ae3-4254-974f-f9cbd00b11af admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:22.651430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82699987-e1b6-4854-86c1-28e6a16b45c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:22.652183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82699987-e1b6-4854-86c1-28e6a16b45c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:22.652530 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82699987-e1b6-4854-86c1-28e6a16b45c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:22.652864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82699987-e1b6-4854-86c1-28e6a16b45c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:22.653567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82699987-e1b6-4854-86c1-28e6a16b45c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:22.655550 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82699987-e1b6-4854-86c1-28e6a16b45c3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:22.676475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1063/2103] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:22 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:17:22.686130 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c76ec3-88cc-4d49-9f4f-29e2195c23a7 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:22.697521 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c76ec3-88cc-4d49-9f4f-29e2195c23a7 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:22.703255 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3517e-5a77-4c48-bd24-54279a42e2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:22.703972 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3517e-5a77-4c48-bd24-54279a42e2d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:22.704259 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3517e-5a77-4c48-bd24-54279a42e2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:22.704537 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3517e-5a77-4c48-bd24-54279a42e2d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:22.705276 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a3517e-5a77-4c48-bd24-54279a42e2d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:22.706913 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a3517e-5a77-4c48-bd24-54279a42e2d0 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:22.719432 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1041/2104] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:22.723876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82699987-e1b6-4854-86c1-28e6a16b45c3 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:22.735215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82699987-e1b6-4854-86c1-28e6a16b45c3 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:22.740588 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51394ce-4fbc-427a-9130-de9eaf61391f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:22.741292 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51394ce-4fbc-427a-9130-de9eaf61391f admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:22.741577 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51394ce-4fbc-427a-9130-de9eaf61391f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:22.741855 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51394ce-4fbc-427a-9130-de9eaf61391f admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:22.743463 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51394ce-4fbc-427a-9130-de9eaf61391f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:22.745187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51394ce-4fbc-427a-9130-de9eaf61391f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:22.757281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1064/2105] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:22.761787 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3517e-5a77-4c48-bd24-54279a42e2d0 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:22.773461 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3517e-5a77-4c48-bd24-54279a42e2d0 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:22.779184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b1db3-33ae-4e30-8e0a-7398e1d8e41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:22.780195 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b1db3-33ae-4e30-8e0a-7398e1d8e41c admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:22.780484 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b1db3-33ae-4e30-8e0a-7398e1d8e41c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:22.780764 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b1db3-33ae-4e30-8e0a-7398e1d8e41c admin admin] PATH_INFO: `/v3/projects/9c06e827e8c44f58a2051705a46e361c/users/92c1e4ea6ab84f94a2215f8c534ca159/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:22.799784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b1db3-33ae-4e30-8e0a-7398e1d8e41c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c06e827e8c44f58a2051705a46e361c, user_id=92c1e4ea6ab84f94a2215f8c534ca159, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:22.802540 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b1db3-33ae-4e30-8e0a-7398e1d8e41c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:22.817283 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1042/2106] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:22 2022] PUT /identity/v3/projects/9c06e827e8c44f58a2051705a46e361c/users/92c1e4ea6ab84f94a2215f8c534ca159/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:17:22.822563 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6573b9f-0b9c-4d1e-b149-6ab7f8ea705b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:22.822881 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6573b9f-0b9c-4d1e-b149-6ab7f8ea705b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:22.823164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6573b9f-0b9c-4d1e-b149-6ab7f8ea705b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:22.841522 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a6573b9f-0b9c-4d1e-b149-6ab7f8ea705b None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:22.842731 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a6573b9f-0b9c-4d1e-b149-6ab7f8ea705b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:22.909447 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1065/2107] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:22.918622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b1db3-33ae-4e30-8e0a-7398e1d8e41c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:22.920989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fb8b1db3-33ae-4e30-8e0a-7398e1d8e41c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:23.002882 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b1db3-33ae-4e30-8e0a-7398e1d8e41c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:23.007458 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda050f-7503-4e63-992f-a81ede356ba0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:23.008085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda050f-7503-4e63-992f-a81ede356ba0 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:23.008377 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda050f-7503-4e63-992f-a81ede356ba0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:23.008694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda050f-7503-4e63-992f-a81ede356ba0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:23.024401 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda050f-7503-4e63-992f-a81ede356ba0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:23.025911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda050f-7503-4e63-992f-a81ede356ba0 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:23.041576 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1043/2108] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:17:23.312791 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6573b9f-0b9c-4d1e-b149-6ab7f8ea705b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:23.314238 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a6573b9f-0b9c-4d1e-b149-6ab7f8ea705b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:23.326348 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6573b9f-0b9c-4d1e-b149-6ab7f8ea705b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:23.331258 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becbdde2-ee48-4cab-81d1-75dad6da729a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:23.332172 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbdde2-ee48-4cab-81d1-75dad6da729a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:23.332450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbdde2-ee48-4cab-81d1-75dad6da729a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:23.332718 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbdde2-ee48-4cab-81d1-75dad6da729a admin admin] PATH_INFO: `/v3/users/99e8411bc3914e37976017a3db7403a1` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:23.333988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becbdde2-ee48-4cab-81d1-75dad6da729a admin admin] RBAC: Authorizing `identity:delete_user(user_id=99e8411bc3914e37976017a3db7403a1)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:23.337223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becbdde2-ee48-4cab-81d1-75dad6da729a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:23.390775 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-becbdde2-ee48-4cab-81d1-75dad6da729a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99e8411bc3914e37976017a3db7403a1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:23.391787 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-becbdde2-ee48-4cab-81d1-75dad6da729a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99e8411bc3914e37976017a3db7403a1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:23.392406 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-becbdde2-ee48-4cab-81d1-75dad6da729a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99e8411bc3914e37976017a3db7403a1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:23.409772 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-becbdde2-ee48-4cab-81d1-75dad6da729a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99e8411bc3914e37976017a3db7403a1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:23.416603 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:23.444340 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-becbdde2-ee48-4cab-81d1-75dad6da729a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99e8411bc3914e37976017a3db7403a1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:23.470125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1066/2109] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:23 2022] DELETE /identity/v3/users/99e8411bc3914e37976017a3db7403a1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:23.475406 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d8444-96c0-4f5f-b81e-e779e46d3454 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:23.475807 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d8444-96c0-4f5f-b81e-e779e46d3454 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:23.476152 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d8444-96c0-4f5f-b81e-e779e46d3454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:23.502095 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-c78d8444-96c0-4f5f-b81e-e779e46d3454 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:23.504737 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c78d8444-96c0-4f5f-b81e-e779e46d3454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:23.605005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1044/2110] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:23.616917 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becbdde2-ee48-4cab-81d1-75dad6da729a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:23.619072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-becbdde2-ee48-4cab-81d1-75dad6da729a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:23.687895 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becbdde2-ee48-4cab-81d1-75dad6da729a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:23.692039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae48994-a412-41d7-b0eb-6efcf11bffb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:23.692767 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae48994-a412-41d7-b0eb-6efcf11bffb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:23.693110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae48994-a412-41d7-b0eb-6efcf11bffb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:23.693548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae48994-a412-41d7-b0eb-6efcf11bffb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:23.713511 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae48994-a412-41d7-b0eb-6efcf11bffb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:23.716122 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae48994-a412-41d7-b0eb-6efcf11bffb8 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:23.729134 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1067/2111] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:24.177115 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d8444-96c0-4f5f-b81e-e779e46d3454 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:24.180091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c78d8444-96c0-4f5f-b81e-e779e46d3454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:24.215073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d8444-96c0-4f5f-b81e-e779e46d3454 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:24.223938 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63d945-ee2d-49fd-b364-892e6a3fbe1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:24.225018 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63d945-ee2d-49fd-b364-892e6a3fbe1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:24.225462 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63d945-ee2d-49fd-b364-892e6a3fbe1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:24.225878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63d945-ee2d-49fd-b364-892e6a3fbe1c admin admin] PATH_INFO: `/v3/projects/dc37ee81e325473c8d8b94d86690e098` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:24.227469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63d945-ee2d-49fd-b364-892e6a3fbe1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc37ee81e325473c8d8b94d86690e098)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:24.229824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63d945-ee2d-49fd-b364-892e6a3fbe1c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:24.279113 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:24.305174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-af63d945-ee2d-49fd-b364-892e6a3fbe1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc37ee81e325473c8d8b94d86690e098', 'request_id': 'req-af63d945-ee2d-49fd-b364-892e6a3fbe1c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:24.315245 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-af63d945-ee2d-49fd-b364-892e6a3fbe1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc37ee81e325473c8d8b94d86690e098', 'request_id': 'req-af63d945-ee2d-49fd-b364-892e6a3fbe1c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:24.323657 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-af63d945-ee2d-49fd-b364-892e6a3fbe1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc37ee81e325473c8d8b94d86690e098', 'request_id': 'req-af63d945-ee2d-49fd-b364-892e6a3fbe1c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:24.339547 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-af63d945-ee2d-49fd-b364-892e6a3fbe1c admin admin] The token cache is being invalidate because project dc37ee81e325473c8d8b94d86690e098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:24.340268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-af63d945-ee2d-49fd-b364-892e6a3fbe1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:24.341244 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-af63d945-ee2d-49fd-b364-892e6a3fbe1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:24.343247 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1045/2112] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:24 2022] DELETE /identity/v3/projects/dc37ee81e325473c8d8b94d86690e098 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:24.348435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae48994-a412-41d7-b0eb-6efcf11bffb8 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:24.360997 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae48994-a412-41d7-b0eb-6efcf11bffb8 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:24.443526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e2df4-65c0-41a2-9905-68f0a28945e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:24.445085 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e2df4-65c0-41a2-9905-68f0a28945e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:24.445475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e2df4-65c0-41a2-9905-68f0a28945e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:24.445852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e2df4-65c0-41a2-9905-68f0a28945e5 admin admin] PATH_INFO: `/v3/users/e072b0de4cc64a03acaa4ab8742b5de4` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:24.447959 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e2df4-65c0-41a2-9905-68f0a28945e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e072b0de4cc64a03acaa4ab8742b5de4)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:24.450274 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e2df4-65c0-41a2-9905-68f0a28945e5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:24.526951 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e84e2df4-65c0-41a2-9905-68f0a28945e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e072b0de4cc64a03acaa4ab8742b5de4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:24.526951 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e84e2df4-65c0-41a2-9905-68f0a28945e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e072b0de4cc64a03acaa4ab8742b5de4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:24.527532 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e84e2df4-65c0-41a2-9905-68f0a28945e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e072b0de4cc64a03acaa4ab8742b5de4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:24.541848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e84e2df4-65c0-41a2-9905-68f0a28945e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e072b0de4cc64a03acaa4ab8742b5de4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:24.550331 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e84e2df4-65c0-41a2-9905-68f0a28945e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e072b0de4cc64a03acaa4ab8742b5de4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:24.574581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1068/2113] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:24 2022] DELETE /identity/v3/users/e072b0de4cc64a03acaa4ab8742b5de4 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:24.948363 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63d945-ee2d-49fd-b364-892e6a3fbe1c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:24.949334 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-af63d945-ee2d-49fd-b364-892e6a3fbe1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:24.971461 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63d945-ee2d-49fd-b364-892e6a3fbe1c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:25.073433 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88a246-fc7f-44f9-8ab9-5ac877feebe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:25.074471 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88a246-fc7f-44f9-8ab9-5ac877feebe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.074851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88a246-fc7f-44f9-8ab9-5ac877feebe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.075187 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88a246-fc7f-44f9-8ab9-5ac877feebe3 admin admin] PATH_INFO: `/v3/projects/60a0162eac2b41edb17e7cefbe96526f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.076617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee88a246-fc7f-44f9-8ab9-5ac877feebe3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60a0162eac2b41edb17e7cefbe96526f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:25.078678 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee88a246-fc7f-44f9-8ab9-5ac877feebe3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:25.136381 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ee88a246-fc7f-44f9-8ab9-5ac877feebe3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60a0162eac2b41edb17e7cefbe96526f', 'request_id': 'req-ee88a246-fc7f-44f9-8ab9-5ac877feebe3'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:25.145420 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ee88a246-fc7f-44f9-8ab9-5ac877feebe3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60a0162eac2b41edb17e7cefbe96526f', 'request_id': 'req-ee88a246-fc7f-44f9-8ab9-5ac877feebe3'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:25.151968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ee88a246-fc7f-44f9-8ab9-5ac877feebe3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60a0162eac2b41edb17e7cefbe96526f', 'request_id': 'req-ee88a246-fc7f-44f9-8ab9-5ac877feebe3'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:25.164710 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ee88a246-fc7f-44f9-8ab9-5ac877feebe3 admin admin] The token cache is being invalidate because project 60a0162eac2b41edb17e7cefbe96526f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:25.165096 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ee88a246-fc7f-44f9-8ab9-5ac877feebe3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:25.165810 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ee88a246-fc7f-44f9-8ab9-5ac877feebe3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:25.167419 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1046/2114] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:24 2022] DELETE /identity/v3/projects/60a0162eac2b41edb17e7cefbe96526f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:25.172636 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fbe72-4715-4399-94a4-1ebfbdebd873 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.173202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fbe72-4715-4399-94a4-1ebfbdebd873 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.173202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fbe72-4715-4399-94a4-1ebfbdebd873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.191940 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9b3fbe72-4715-4399-94a4-1ebfbdebd873 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:25.193215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9b3fbe72-4715-4399-94a4-1ebfbdebd873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:25.198249 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:25.262747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1069/2115] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:25.271128 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa30897-496e-4766-8c2f-b2f14d01ba40 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.271727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa30897-496e-4766-8c2f-b2f14d01ba40 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.271727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa30897-496e-4766-8c2f-b2f14d01ba40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.294313 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-dfa30897-496e-4766-8c2f-b2f14d01ba40 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:25.297320 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-dfa30897-496e-4766-8c2f-b2f14d01ba40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:25.305846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1047/2116] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:25.310144 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fbe72-4715-4399-94a4-1ebfbdebd873 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:25.320238 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fbe72-4715-4399-94a4-1ebfbdebd873 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:25.325004 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65da926-298b-49fe-aad9-052060bba84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:25.325749 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65da926-298b-49fe-aad9-052060bba84b admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.326033 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65da926-298b-49fe-aad9-052060bba84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.326304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65da926-298b-49fe-aad9-052060bba84b admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.327036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65da926-298b-49fe-aad9-052060bba84b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:25.327342 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65da926-298b-49fe-aad9-052060bba84b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:25.329011 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65da926-298b-49fe-aad9-052060bba84b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:25.341860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1070/2117] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:17:25.346493 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa30897-496e-4766-8c2f-b2f14d01ba40 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:25.359439 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa30897-496e-4766-8c2f-b2f14d01ba40 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:25.365500 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a0ed1-d455-40a0-8df1-4cfab9253bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:25.366737 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a0ed1-d455-40a0-8df1-4cfab9253bed admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.367231 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a0ed1-d455-40a0-8df1-4cfab9253bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.367764 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a0ed1-d455-40a0-8df1-4cfab9253bed admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.369496 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a0ed1-d455-40a0-8df1-4cfab9253bed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:25.372537 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a0ed1-d455-40a0-8df1-4cfab9253bed admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:25.404383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1048/2118] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:25 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:25.409710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65da926-298b-49fe-aad9-052060bba84b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:25.429475 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65da926-298b-49fe-aad9-052060bba84b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:25.492483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0f6ee-0f89-46fd-b145-971f6c9ce5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:25.493366 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0f6ee-0f89-46fd-b145-971f6c9ce5de admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.493747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0f6ee-0f89-46fd-b145-971f6c9ce5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.494277 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0f6ee-0f89-46fd-b145-971f6c9ce5de admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.494972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0f6ee-0f89-46fd-b145-971f6c9ce5de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:25.497014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0f6ee-0f89-46fd-b145-971f6c9ce5de admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:25.520381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1071/2119] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:25 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:25.525026 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a0ed1-d455-40a0-8df1-4cfab9253bed admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:25.539565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a0ed1-d455-40a0-8df1-4cfab9253bed admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:25.545255 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca97f3f-84ba-4b16-8737-08631dea37fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:25.545952 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca97f3f-84ba-4b16-8737-08631dea37fc admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.546236 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca97f3f-84ba-4b16-8737-08631dea37fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.546507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca97f3f-84ba-4b16-8737-08631dea37fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.547237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca97f3f-84ba-4b16-8737-08631dea37fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:25.548926 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca97f3f-84ba-4b16-8737-08631dea37fc admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:25.566522 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1049/2120] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:25.570297 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0f6ee-0f89-46fd-b145-971f6c9ce5de admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:25.581299 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0f6ee-0f89-46fd-b145-971f6c9ce5de admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:25.586512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c113fd-2eb9-4797-9ae3-d058a1dfea78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:25.587312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c113fd-2eb9-4797-9ae3-d058a1dfea78 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.587775 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c113fd-2eb9-4797-9ae3-d058a1dfea78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.588156 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c113fd-2eb9-4797-9ae3-d058a1dfea78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.589060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c113fd-2eb9-4797-9ae3-d058a1dfea78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:25.590916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c113fd-2eb9-4797-9ae3-d058a1dfea78 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:25.606315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1072/2121] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:25.610287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca97f3f-84ba-4b16-8737-08631dea37fc admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:25.617425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:25.621821 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca97f3f-84ba-4b16-8737-08631dea37fc admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:25.628102 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c36eea-ae8d-4f4f-9171-cf3d9a547813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:25.629307 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c36eea-ae8d-4f4f-9171-cf3d9a547813 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.629772 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c36eea-ae8d-4f4f-9171-cf3d9a547813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.630212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c36eea-ae8d-4f4f-9171-cf3d9a547813 admin admin] PATH_INFO: `/v3/projects/cb85279246c142a98e519de1db70c610/users/8df15a5fbf8e498ba02552c2e24a3dda/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.649927 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c36eea-ae8d-4f4f-9171-cf3d9a547813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb85279246c142a98e519de1db70c610, user_id=8df15a5fbf8e498ba02552c2e24a3dda, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:25.652730 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c36eea-ae8d-4f4f-9171-cf3d9a547813 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:25.668589 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1050/2122] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:25 2022] PUT /identity/v3/projects/cb85279246c142a98e519de1db70c610/users/8df15a5fbf8e498ba02552c2e24a3dda/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:17:25.673637 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a528b-805c-4fe1-be7f-da182fc21990 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.673980 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a528b-805c-4fe1-be7f-da182fc21990 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.674335 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a528b-805c-4fe1-be7f-da182fc21990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.706425 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0d1a528b-805c-4fe1-be7f-da182fc21990 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:25.707666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0d1a528b-805c-4fe1-be7f-da182fc21990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:25.785122 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1073/2123] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:25.796181 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c36eea-ae8d-4f4f-9171-cf3d9a547813 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:25.799927 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c4c36eea-ae8d-4f4f-9171-cf3d9a547813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:25.876942 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c36eea-ae8d-4f4f-9171-cf3d9a547813 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:25.880697 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac83c1-212b-4875-b37f-f6002a239c4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:25.881380 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac83c1-212b-4875-b37f-f6002a239c4b service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.881666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac83c1-212b-4875-b37f-f6002a239c4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.881944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac83c1-212b-4875-b37f-f6002a239c4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.893052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac83c1-212b-4875-b37f-f6002a239c4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:25.894524 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac83c1-212b-4875-b37f-f6002a239c4b service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:25.905587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1051/2124] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:26.793222 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff15b85-53f2-4295-83e5-a59742a730ce None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:26.793694 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff15b85-53f2-4295-83e5-a59742a730ce None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:26.793931 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff15b85-53f2-4295-83e5-a59742a730ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:26.826285 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7ff15b85-53f2-4295-83e5-a59742a730ce None None] MFA Rules not processed for user `8df15a5fbf8e498ba02552c2e24a3dda`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:26.827866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7ff15b85-53f2-4295-83e5-a59742a730ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:26.832152 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:26.933720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1074/2125] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:26.938398 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac83c1-212b-4875-b37f-f6002a239c4b service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:26.951520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac83c1-212b-4875-b37f-f6002a239c4b service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:26.961739 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808a702-37e2-4f1c-afc9-df72800cb59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:26.962955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808a702-37e2-4f1c-afc9-df72800cb59c admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:26.963359 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808a702-37e2-4f1c-afc9-df72800cb59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:26.963584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808a702-37e2-4f1c-afc9-df72800cb59c admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:26.964548 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808a702-37e2-4f1c-afc9-df72800cb59c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:26.966734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808a702-37e2-4f1c-afc9-df72800cb59c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:27.010362 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1052/2126] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:26 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:27.013001 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff15b85-53f2-4295-83e5-a59742a730ce None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:27.034524 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff15b85-53f2-4295-83e5-a59742a730ce None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:27.137021 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a891271-58d9-4556-aac5-16da1b0d588d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:27.137638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a891271-58d9-4556-aac5-16da1b0d588d admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:27.137919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a891271-58d9-4556-aac5-16da1b0d588d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:27.138199 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a891271-58d9-4556-aac5-16da1b0d588d admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:27.138899 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a891271-58d9-4556-aac5-16da1b0d588d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:27.140893 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a891271-58d9-4556-aac5-16da1b0d588d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:27.169557 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1075/2127] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:27 2022] POST /identity/v3/users => generated 292 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:27.173726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808a702-37e2-4f1c-afc9-df72800cb59c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:27.186831 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808a702-37e2-4f1c-afc9-df72800cb59c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:27.191986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac7590-1b87-405b-8d3c-d1dd9d0f4643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:27.193049 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac7590-1b87-405b-8d3c-d1dd9d0f4643 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:27.193474 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac7590-1b87-405b-8d3c-d1dd9d0f4643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:27.193925 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac7590-1b87-405b-8d3c-d1dd9d0f4643 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:27.195030 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac7590-1b87-405b-8d3c-d1dd9d0f4643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:27.196906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac7590-1b87-405b-8d3c-d1dd9d0f4643 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:27.210135 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1053/2128] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:27.214075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a891271-58d9-4556-aac5-16da1b0d588d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:27.226206 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a891271-58d9-4556-aac5-16da1b0d588d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:27.231551 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2090bc0-440c-4dee-b3c4-1e440f0bd635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:27.232517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2090bc0-440c-4dee-b3c4-1e440f0bd635 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:27.232844 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2090bc0-440c-4dee-b3c4-1e440f0bd635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:27.233134 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2090bc0-440c-4dee-b3c4-1e440f0bd635 admin admin] PATH_INFO: `/v3/projects/663853349fcc4b0f9c969d4e98300685/users/02bcbca6c4e9446c9f00bdf2cd174fad/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:27.252783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2090bc0-440c-4dee-b3c4-1e440f0bd635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=663853349fcc4b0f9c969d4e98300685, user_id=02bcbca6c4e9446c9f00bdf2cd174fad, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:27.255710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2090bc0-440c-4dee-b3c4-1e440f0bd635 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:27.275991 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1076/2129] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:27 2022] PUT /identity/v3/projects/663853349fcc4b0f9c969d4e98300685/users/02bcbca6c4e9446c9f00bdf2cd174fad/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:27.279637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac7590-1b87-405b-8d3c-d1dd9d0f4643 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:27.293963 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac7590-1b87-405b-8d3c-d1dd9d0f4643 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:27.379146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae72f2-0658-45b4-b32f-0168eac36e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:27.380096 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae72f2-0658-45b4-b32f-0168eac36e91 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:27.380460 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae72f2-0658-45b4-b32f-0168eac36e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:27.380799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae72f2-0658-45b4-b32f-0168eac36e91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:27.381618 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae72f2-0658-45b4-b32f-0168eac36e91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:27.383349 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae72f2-0658-45b4-b32f-0168eac36e91 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:27.396263 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1054/2130] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:27.400775 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2090bc0-440c-4dee-b3c4-1e440f0bd635 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:27.416805 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2090bc0-440c-4dee-b3c4-1e440f0bd635 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:27.428196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4662eeb-ddfc-4fb7-9b78-719b89007649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:27.430810 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4662eeb-ddfc-4fb7-9b78-719b89007649 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:27.431095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4662eeb-ddfc-4fb7-9b78-719b89007649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:27.431812 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4662eeb-ddfc-4fb7-9b78-719b89007649 admin admin] PATH_INFO: `/v3/projects/663853349fcc4b0f9c969d4e98300685/users/02bcbca6c4e9446c9f00bdf2cd174fad/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:27.434051 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4662eeb-ddfc-4fb7-9b78-719b89007649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=663853349fcc4b0f9c969d4e98300685, user_id=02bcbca6c4e9446c9f00bdf2cd174fad, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:27.436802 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4662eeb-ddfc-4fb7-9b78-719b89007649 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:27.461947 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1077/2131] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:27 2022] PUT /identity/v3/projects/663853349fcc4b0f9c969d4e98300685/users/02bcbca6c4e9446c9f00bdf2cd174fad/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:27.735750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc97c1e-84b8-4c26-9786-8a4882da1eb3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:27.736097 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc97c1e-84b8-4c26-9786-8a4882da1eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:27.736385 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc97c1e-84b8-4c26-9786-8a4882da1eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:27.774722 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-2fc97c1e-84b8-4c26-9786-8a4882da1eb3 None None] MFA Rules not processed for user `92c1e4ea6ab84f94a2215f8c534ca159`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:27.776642 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2fc97c1e-84b8-4c26-9786-8a4882da1eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:27.833509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:27.910725 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1055/2132] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:27.920801 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad9b8d-6c70-430c-a152-bd66f38e144b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:27.921838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad9b8d-6c70-430c-a152-bd66f38e144b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:27.922336 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad9b8d-6c70-430c-a152-bd66f38e144b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:27.943204 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-61ad9b8d-6c70-430c-a152-bd66f38e144b None None] MFA Rules not processed for user `92c1e4ea6ab84f94a2215f8c534ca159`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:27.944744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-61ad9b8d-6c70-430c-a152-bd66f38e144b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:27.951856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1078/2133] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:27.962443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc97c1e-84b8-4c26-9786-8a4882da1eb3 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:27.964142 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2fc97c1e-84b8-4c26-9786-8a4882da1eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:28.054239 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc97c1e-84b8-4c26-9786-8a4882da1eb3 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:28.061047 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40440f-1867-4f8b-a896-55744283d5bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:28.061804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40440f-1867-4f8b-a896-55744283d5bd service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:28.062586 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40440f-1867-4f8b-a896-55744283d5bd service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:28.063184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40440f-1867-4f8b-a896-55744283d5bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:28.081718 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40440f-1867-4f8b-a896-55744283d5bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:28.083823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40440f-1867-4f8b-a896-55744283d5bd service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:28.094572 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1056/2134] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:28.402910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a7d57-7f38-44de-9c6b-fb863c9c96f5 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:28.403226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a7d57-7f38-44de-9c6b-fb863c9c96f5 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:28.403626 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a7d57-7f38-44de-9c6b-fb863c9c96f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:28.473437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f08a7d57-7f38-44de-9c6b-fb863c9c96f5 None None] MFA Rules not processed for user `02bcbca6c4e9446c9f00bdf2cd174fad`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:28.474671 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f08a7d57-7f38-44de-9c6b-fb863c9c96f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:28.577734 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:28.634629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1079/2135] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:28 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:28.709680 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40440f-1867-4f8b-a896-55744283d5bd service nova] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:28.711275 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8f40440f-1867-4f8b-a896-55744283d5bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:28.832155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40440f-1867-4f8b-a896-55744283d5bd service nova] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:28.842502 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb67524-c93f-478a-a82d-27739b70adac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:28.843176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb67524-c93f-478a-a82d-27739b70adac service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:28.843456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb67524-c93f-478a-a82d-27739b70adac service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:28.846259 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb67524-c93f-478a-a82d-27739b70adac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:28.991409 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb67524-c93f-478a-a82d-27739b70adac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:28.997348 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb67524-c93f-478a-a82d-27739b70adac service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:29.035811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1057/2136] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:29.057065 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de311bc9-3364-4b00-8aa3-9adca9d49c12 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:29.057572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de311bc9-3364-4b00-8aa3-9adca9d49c12 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:29.057755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de311bc9-3364-4b00-8aa3-9adca9d49c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:29.095129 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-de311bc9-3364-4b00-8aa3-9adca9d49c12 None None] MFA Rules not processed for user `92c1e4ea6ab84f94a2215f8c534ca159`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:29.097114 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-de311bc9-3364-4b00-8aa3-9adca9d49c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:29.107151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1080/2137] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:30.001795 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb67524-c93f-478a-a82d-27739b70adac service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:30.003541 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-adb67524-c93f-478a-a82d-27739b70adac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:30.023283 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:30.078483 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb67524-c93f-478a-a82d-27739b70adac service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:30.084077 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02a80a-ad32-4c93-9db4-ca8554d26457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:30.084985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02a80a-ad32-4c93-9db4-ca8554d26457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:30.085293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02a80a-ad32-4c93-9db4-ca8554d26457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:30.085586 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02a80a-ad32-4c93-9db4-ca8554d26457 admin admin] PATH_INFO: `/v3/users/2fbdcaa712b84abdba95a8eb6c8843da` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:30.086811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d02a80a-ad32-4c93-9db4-ca8554d26457 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fbdcaa712b84abdba95a8eb6c8843da)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:30.089023 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d02a80a-ad32-4c93-9db4-ca8554d26457 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:30.174137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9d02a80a-ad32-4c93-9db4-ca8554d26457 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fbdcaa712b84abdba95a8eb6c8843da'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:30.175062 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9d02a80a-ad32-4c93-9db4-ca8554d26457 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fbdcaa712b84abdba95a8eb6c8843da'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:30.176066 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9d02a80a-ad32-4c93-9db4-ca8554d26457 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fbdcaa712b84abdba95a8eb6c8843da'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:30.195139 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9d02a80a-ad32-4c93-9db4-ca8554d26457 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fbdcaa712b84abdba95a8eb6c8843da'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:30.208556 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9d02a80a-ad32-4c93-9db4-ca8554d26457 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fbdcaa712b84abdba95a8eb6c8843da'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:30.231926 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1058/2138] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:30 2022] DELETE /identity/v3/users/2fbdcaa712b84abdba95a8eb6c8843da => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:30.238935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90de342-f763-4cdf-a63c-481136148cf9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:30.239426 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90de342-f763-4cdf-a63c-481136148cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:30.239528 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90de342-f763-4cdf-a63c-481136148cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:30.272269 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f90de342-f763-4cdf-a63c-481136148cf9 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:30.273519 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f90de342-f763-4cdf-a63c-481136148cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:30.409304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1081/2139] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:30.419294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02a80a-ad32-4c93-9db4-ca8554d26457 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:30.421103 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9d02a80a-ad32-4c93-9db4-ca8554d26457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:30.540382 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02a80a-ad32-4c93-9db4-ca8554d26457 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:30.552329 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3bae33-c1d2-4e92-87c1-dc9eff17a46b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:30.553027 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bae33-c1d2-4e92-87c1-dc9eff17a46b service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:30.553313 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bae33-c1d2-4e92-87c1-dc9eff17a46b service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:30.553590 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bae33-c1d2-4e92-87c1-dc9eff17a46b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:30.577294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3bae33-c1d2-4e92-87c1-dc9eff17a46b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:30.578834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3bae33-c1d2-4e92-87c1-dc9eff17a46b service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:30.604186 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1059/2140] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:17:30.625576 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90de342-f763-4cdf-a63c-481136148cf9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:30.627458 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f90de342-f763-4cdf-a63c-481136148cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:30.725933 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90de342-f763-4cdf-a63c-481136148cf9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:30.729664 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f76db3-cef4-41c1-8aa9-ffd3f0c5cc71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:30.730608 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f76db3-cef4-41c1-8aa9-ffd3f0c5cc71 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:30.731054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f76db3-cef4-41c1-8aa9-ffd3f0c5cc71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:30.731477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f76db3-cef4-41c1-8aa9-ffd3f0c5cc71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:30.741860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a5f76db3-cef4-41c1-8aa9-ffd3f0c5cc71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:30.883490 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f76db3-cef4-41c1-8aa9-ffd3f0c5cc71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:30.885664 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f76db3-cef4-41c1-8aa9-ffd3f0c5cc71 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:30.919639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1082/2141] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:30 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:31.116718 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3bae33-c1d2-4e92-87c1-dc9eff17a46b service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:31.122547 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ef3bae33-c1d2-4e92-87c1-dc9eff17a46b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:31.174257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3bae33-c1d2-4e92-87c1-dc9eff17a46b service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:31.196306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae23b00-4a50-4380-8a35-1b271ecb5749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:31.197361 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae23b00-4a50-4380-8a35-1b271ecb5749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:31.197738 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae23b00-4a50-4380-8a35-1b271ecb5749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:31.198303 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae23b00-4a50-4380-8a35-1b271ecb5749 admin admin] PATH_INFO: `/v3/projects/54cbfe298d34475cbf86b5c0be99acda` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:31.199924 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae23b00-4a50-4380-8a35-1b271ecb5749 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54cbfe298d34475cbf86b5c0be99acda)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:31.203649 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae23b00-4a50-4380-8a35-1b271ecb5749 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:31.267723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:31.289326 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fae23b00-4a50-4380-8a35-1b271ecb5749 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54cbfe298d34475cbf86b5c0be99acda', 'request_id': 'req-fae23b00-4a50-4380-8a35-1b271ecb5749'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:31.298754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fae23b00-4a50-4380-8a35-1b271ecb5749 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54cbfe298d34475cbf86b5c0be99acda', 'request_id': 'req-fae23b00-4a50-4380-8a35-1b271ecb5749'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:31.321889 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fae23b00-4a50-4380-8a35-1b271ecb5749 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54cbfe298d34475cbf86b5c0be99acda', 'request_id': 'req-fae23b00-4a50-4380-8a35-1b271ecb5749'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:31.338539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fae23b00-4a50-4380-8a35-1b271ecb5749 admin admin] The token cache is being invalidate because project 54cbfe298d34475cbf86b5c0be99acda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:31.338828 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fae23b00-4a50-4380-8a35-1b271ecb5749 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:31.339684 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fae23b00-4a50-4380-8a35-1b271ecb5749 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:31.342491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1060/2142] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:31 2022] DELETE /identity/v3/projects/54cbfe298d34475cbf86b5c0be99acda => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:31.347164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6daa89-edc9-4608-9229-900534aaf8f0 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:31.348131 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6daa89-edc9-4608-9229-900534aaf8f0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:31.348550 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6daa89-edc9-4608-9229-900534aaf8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:31.368561 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5a6daa89-edc9-4608-9229-900534aaf8f0 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:31.370097 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5a6daa89-edc9-4608-9229-900534aaf8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:31.389313 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:31.472755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1083/2143] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:31.484795 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61150ad6-6ec1-4e84-972d-6bde1454f8ed None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:31.485088 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61150ad6-6ec1-4e84-972d-6bde1454f8ed None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:31.485375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61150ad6-6ec1-4e84-972d-6bde1454f8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:31.516717 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-61150ad6-6ec1-4e84-972d-6bde1454f8ed None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:31.517617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-61150ad6-6ec1-4e84-972d-6bde1454f8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:31.532913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6daa89-edc9-4608-9229-900534aaf8f0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:31.543341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1061/2144] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:31.547187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6daa89-edc9-4608-9229-900534aaf8f0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:31.552879 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec772e9e-5ba8-470c-a81d-e52c4a2c83eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:31.554320 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec772e9e-5ba8-470c-a81d-e52c4a2c83eb admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:31.554751 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec772e9e-5ba8-470c-a81d-e52c4a2c83eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:31.555126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec772e9e-5ba8-470c-a81d-e52c4a2c83eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:31.558447 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec772e9e-5ba8-470c-a81d-e52c4a2c83eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:31.559037 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec772e9e-5ba8-470c-a81d-e52c4a2c83eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:31.561129 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec772e9e-5ba8-470c-a81d-e52c4a2c83eb admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:31.582648 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1084/2145] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:17:31.583558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61150ad6-6ec1-4e84-972d-6bde1454f8ed None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:31.598900 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61150ad6-6ec1-4e84-972d-6bde1454f8ed None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:31.618130 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1b76e-4fda-45bf-b2c8-7949a3cc0b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:31.620875 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1b76e-4fda-45bf-b2c8-7949a3cc0b47 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:31.622092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1b76e-4fda-45bf-b2c8-7949a3cc0b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:31.622390 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1b76e-4fda-45bf-b2c8-7949a3cc0b47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:31.623129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1b76e-4fda-45bf-b2c8-7949a3cc0b47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:31.629317 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1b76e-4fda-45bf-b2c8-7949a3cc0b47 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:31.698844 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1062/2146] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:31 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:31.701661 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec772e9e-5ba8-470c-a81d-e52c4a2c83eb admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:31.732948 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec772e9e-5ba8-470c-a81d-e52c4a2c83eb admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:31.918263 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e502469-8e4f-4ca8-8850-4a9673386619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:31.919120 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e502469-8e4f-4ca8-8850-4a9673386619 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:31.919412 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e502469-8e4f-4ca8-8850-4a9673386619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:31.919719 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e502469-8e4f-4ca8-8850-4a9673386619 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:31.920482 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e502469-8e4f-4ca8-8850-4a9673386619 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:31.933580 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e502469-8e4f-4ca8-8850-4a9673386619 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:31.977446 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1085/2147] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:31 2022] POST /identity/v3/users => generated 286 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:31.978471 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1b76e-4fda-45bf-b2c8-7949a3cc0b47 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:31.993897 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1b76e-4fda-45bf-b2c8-7949a3cc0b47 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:32.010721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271a1d7-18d5-418e-a1da-105325d8391d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:32.011784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271a1d7-18d5-418e-a1da-105325d8391d admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:32.012181 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271a1d7-18d5-418e-a1da-105325d8391d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:32.012470 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271a1d7-18d5-418e-a1da-105325d8391d admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:32.013198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271a1d7-18d5-418e-a1da-105325d8391d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:32.028411 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271a1d7-18d5-418e-a1da-105325d8391d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:32.054494 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e502469-8e4f-4ca8-8850-4a9673386619 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:32.066056 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1063/2148] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:17:32.068717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e502469-8e4f-4ca8-8850-4a9673386619 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:32.075636 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21104a9-2fc5-45ea-bec6-0080286fc95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:32.076642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21104a9-2fc5-45ea-bec6-0080286fc95f admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:32.077526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21104a9-2fc5-45ea-bec6-0080286fc95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:32.078132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21104a9-2fc5-45ea-bec6-0080286fc95f admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:32.079048 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21104a9-2fc5-45ea-bec6-0080286fc95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:32.081743 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21104a9-2fc5-45ea-bec6-0080286fc95f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:32.104840 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1086/2149] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:32.109330 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271a1d7-18d5-418e-a1da-105325d8391d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:32.121748 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271a1d7-18d5-418e-a1da-105325d8391d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:32.126786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbef3b-cb29-41f2-908c-c06f3dd9aa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:32.128261 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbef3b-cb29-41f2-908c-c06f3dd9aa0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:32.128563 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbef3b-cb29-41f2-908c-c06f3dd9aa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:32.128843 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbef3b-cb29-41f2-908c-c06f3dd9aa0e admin admin] PATH_INFO: `/v3/projects/27b2158e24b0466c9b344c4f8c4f1301/users/cc12ebd7d75b460f9e520efcbad1aa11/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:32.154627 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbef3b-cb29-41f2-908c-c06f3dd9aa0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b2158e24b0466c9b344c4f8c4f1301, user_id=cc12ebd7d75b460f9e520efcbad1aa11, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:32.157680 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbef3b-cb29-41f2-908c-c06f3dd9aa0e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:32.177856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1064/2150] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:32 2022] PUT /identity/v3/projects/27b2158e24b0466c9b344c4f8c4f1301/users/cc12ebd7d75b460f9e520efcbad1aa11/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:32.189281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0d7d9-8d62-4f86-a28d-bb4c4efc9ffd None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:32.189685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0d7d9-8d62-4f86-a28d-bb4c4efc9ffd None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:32.190053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0d7d9-8d62-4f86-a28d-bb4c4efc9ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:32.259791 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6db0d7d9-8d62-4f86-a28d-bb4c4efc9ffd None None] MFA Rules not processed for user `cc12ebd7d75b460f9e520efcbad1aa11`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:32.261385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6db0d7d9-8d62-4f86-a28d-bb4c4efc9ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:32.390794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1087/2151] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:32.407007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbef3b-cb29-41f2-908c-c06f3dd9aa0e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:32.409426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a9cbef3b-cb29-41f2-908c-c06f3dd9aa0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:32.628216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbef3b-cb29-41f2-908c-c06f3dd9aa0e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:32.639426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581deb84-4ffd-4848-a1d1-a22b357ef9b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:32.640329 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581deb84-4ffd-4848-a1d1-a22b357ef9b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:32.640713 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581deb84-4ffd-4848-a1d1-a22b357ef9b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:32.641449 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581deb84-4ffd-4848-a1d1-a22b357ef9b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:32.669209 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581deb84-4ffd-4848-a1d1-a22b357ef9b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:32.675985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581deb84-4ffd-4848-a1d1-a22b357ef9b8 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:32.692490 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1065/2152] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:17:34.519466 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42e3d1-58f7-4b86-b4ae-a3cc1cc055e8 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:34.519963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42e3d1-58f7-4b86-b4ae-a3cc1cc055e8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:34.520323 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42e3d1-58f7-4b86-b4ae-a3cc1cc055e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:34.552568 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ed42e3d1-58f7-4b86-b4ae-a3cc1cc055e8 None None] MFA Rules not processed for user `cc12ebd7d75b460f9e520efcbad1aa11`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:34.553989 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ed42e3d1-58f7-4b86-b4ae-a3cc1cc055e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:34.566427 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1088/2153] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:38.701661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581deb84-4ffd-4848-a1d1-a22b357ef9b8 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:38.703288 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-581deb84-4ffd-4848-a1d1-a22b357ef9b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:38.779977 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581deb84-4ffd-4848-a1d1-a22b357ef9b8 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:38.784408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0302f-d871-419b-b830-bbef255fb7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:38.785293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0302f-d871-419b-b830-bbef255fb7b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:38.785568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0302f-d871-419b-b830-bbef255fb7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:38.785837 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0302f-d871-419b-b830-bbef255fb7b4 admin admin] PATH_INFO: `/v3/users/a99128300cf7482abd8ebb038b4ed0bc` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:38.787015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0302f-d871-419b-b830-bbef255fb7b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a99128300cf7482abd8ebb038b4ed0bc)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:38.788997 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0302f-d871-419b-b830-bbef255fb7b4 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:38.790821 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:38.846240 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bed0302f-d871-419b-b830-bbef255fb7b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a99128300cf7482abd8ebb038b4ed0bc'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:38.846896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bed0302f-d871-419b-b830-bbef255fb7b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a99128300cf7482abd8ebb038b4ed0bc'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:38.847521 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bed0302f-d871-419b-b830-bbef255fb7b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a99128300cf7482abd8ebb038b4ed0bc'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:38.861324 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bed0302f-d871-419b-b830-bbef255fb7b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a99128300cf7482abd8ebb038b4ed0bc'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:38.869783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bed0302f-d871-419b-b830-bbef255fb7b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a99128300cf7482abd8ebb038b4ed0bc'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:38.884972 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1066/2154] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:38 2022] DELETE /identity/v3/users/a99128300cf7482abd8ebb038b4ed0bc => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:38.889963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb14a59-d427-473d-9195-f46a8a428680 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:38.890444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb14a59-d427-473d-9195-f46a8a428680 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:38.890628 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb14a59-d427-473d-9195-f46a8a428680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:38.919427 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ecb14a59-d427-473d-9195-f46a8a428680 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:38.920959 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ecb14a59-d427-473d-9195-f46a8a428680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:38.925792 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:38.993419 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1089/2155] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:39.003708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0302f-d871-419b-b830-bbef255fb7b4 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:39.005132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-bed0302f-d871-419b-b830-bbef255fb7b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:39.064627 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0302f-d871-419b-b830-bbef255fb7b4 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:39.068268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867b88f-a0ec-4d26-a1b5-89f19b7dc099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:39.068918 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867b88f-a0ec-4d26-a1b5-89f19b7dc099 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:39.069198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867b88f-a0ec-4d26-a1b5-89f19b7dc099 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:39.069469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867b88f-a0ec-4d26-a1b5-89f19b7dc099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:39.078900 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867b88f-a0ec-4d26-a1b5-89f19b7dc099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:39.080370 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867b88f-a0ec-4d26-a1b5-89f19b7dc099 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:39.090091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1067/2156] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:39.584374 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb14a59-d427-473d-9195-f46a8a428680 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:39.586506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ecb14a59-d427-473d-9195-f46a8a428680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:39.603512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb14a59-d427-473d-9195-f46a8a428680 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:39.629847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1b3cd-67eb-4129-aed9-291e1dafe973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:39.631089 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1b3cd-67eb-4129-aed9-291e1dafe973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:39.631457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1b3cd-67eb-4129-aed9-291e1dafe973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:39.631835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1b3cd-67eb-4129-aed9-291e1dafe973 admin admin] PATH_INFO: `/v3/projects/d7be01501aa04a6297298f15391ecd4d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:39.633739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1b3cd-67eb-4129-aed9-291e1dafe973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7be01501aa04a6297298f15391ecd4d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:39.638846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1b3cd-67eb-4129-aed9-291e1dafe973 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:39.721631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c7f1b3cd-67eb-4129-aed9-291e1dafe973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7be01501aa04a6297298f15391ecd4d', 'request_id': 'req-c7f1b3cd-67eb-4129-aed9-291e1dafe973'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:39.733113 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c7f1b3cd-67eb-4129-aed9-291e1dafe973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7be01501aa04a6297298f15391ecd4d', 'request_id': 'req-c7f1b3cd-67eb-4129-aed9-291e1dafe973'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:39.754109 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c7f1b3cd-67eb-4129-aed9-291e1dafe973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7be01501aa04a6297298f15391ecd4d', 'request_id': 'req-c7f1b3cd-67eb-4129-aed9-291e1dafe973'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:39.773426 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c7f1b3cd-67eb-4129-aed9-291e1dafe973 admin admin] The token cache is being invalidate because project d7be01501aa04a6297298f15391ecd4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:39.773987 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c7f1b3cd-67eb-4129-aed9-291e1dafe973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:39.774865 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c7f1b3cd-67eb-4129-aed9-291e1dafe973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:39.780774 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1090/2157] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:39 2022] DELETE /identity/v3/projects/d7be01501aa04a6297298f15391ecd4d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:39.782508 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f0d01-9640-4c43-aded-92508aa8904b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:39.783020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f0d01-9640-4c43-aded-92508aa8904b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:39.783371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f0d01-9640-4c43-aded-92508aa8904b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:39.820483 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e31f0d01-9640-4c43-aded-92508aa8904b None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:39.822858 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e31f0d01-9640-4c43-aded-92508aa8904b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:39.909577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1068/2158] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:39.917977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a09ba-dfca-4617-b4cf-5dbcaa363ff2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:39.918920 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a09ba-dfca-4617-b4cf-5dbcaa363ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:39.919333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a09ba-dfca-4617-b4cf-5dbcaa363ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:39.958026 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-311a09ba-dfca-4617-b4cf-5dbcaa363ff2 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:39.959685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-311a09ba-dfca-4617-b4cf-5dbcaa363ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:40.008837 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1091/2159] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:40.013812 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f0d01-9640-4c43-aded-92508aa8904b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:40.027470 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f0d01-9640-4c43-aded-92508aa8904b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:40.042552 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247b1e1-5627-4865-a9aa-3da766ab6414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:40.043326 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247b1e1-5627-4865-a9aa-3da766ab6414 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:40.043727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247b1e1-5627-4865-a9aa-3da766ab6414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:40.044076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247b1e1-5627-4865-a9aa-3da766ab6414 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:40.045070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247b1e1-5627-4865-a9aa-3da766ab6414 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:40.045560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247b1e1-5627-4865-a9aa-3da766ab6414 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:40.048408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247b1e1-5627-4865-a9aa-3da766ab6414 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:40.066770 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1069/2160] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:17:40.069368 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a09ba-dfca-4617-b4cf-5dbcaa363ff2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:40.087776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a09ba-dfca-4617-b4cf-5dbcaa363ff2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:40.095172 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c158e-7e56-4296-9edb-5d8cd9e5506e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:40.096188 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c158e-7e56-4296-9edb-5d8cd9e5506e admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:40.097290 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c158e-7e56-4296-9edb-5d8cd9e5506e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:40.097782 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c158e-7e56-4296-9edb-5d8cd9e5506e admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:40.099337 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c158e-7e56-4296-9edb-5d8cd9e5506e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:40.102272 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c158e-7e56-4296-9edb-5d8cd9e5506e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:40.144358 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1092/2161] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:40 2022] POST /identity/v3/projects => generated 389 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:40.145567 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247b1e1-5627-4865-a9aa-3da766ab6414 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:40.156216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247b1e1-5627-4865-a9aa-3da766ab6414 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:40.161290 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:40.219017 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9b714-31fb-435f-9b4c-aac2d7e5bd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:40.219627 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9b714-31fb-435f-9b4c-aac2d7e5bd39 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:40.219921 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9b714-31fb-435f-9b4c-aac2d7e5bd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:40.220193 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9b714-31fb-435f-9b4c-aac2d7e5bd39 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:40.220884 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9b714-31fb-435f-9b4c-aac2d7e5bd39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:40.222810 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9b714-31fb-435f-9b4c-aac2d7e5bd39 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:40.243255 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1070/2162] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:40 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:40.247300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c158e-7e56-4296-9edb-5d8cd9e5506e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:40.257761 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c158e-7e56-4296-9edb-5d8cd9e5506e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:40.263874 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039fb729-dbcd-4db4-8658-537d3deaeb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:40.264863 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039fb729-dbcd-4db4-8658-537d3deaeb10 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:40.265323 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039fb729-dbcd-4db4-8658-537d3deaeb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:40.265746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039fb729-dbcd-4db4-8658-537d3deaeb10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:40.266997 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039fb729-dbcd-4db4-8658-537d3deaeb10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:40.268694 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039fb729-dbcd-4db4-8658-537d3deaeb10 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:40.285050 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1093/2163] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:40.286174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9b714-31fb-435f-9b4c-aac2d7e5bd39 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:40.298808 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9b714-31fb-435f-9b4c-aac2d7e5bd39 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:40.306077 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393df39a-383c-48b2-8f64-55a5bea6535b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:40.307250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393df39a-383c-48b2-8f64-55a5bea6535b admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:40.307533 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393df39a-383c-48b2-8f64-55a5bea6535b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:40.307841 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393df39a-383c-48b2-8f64-55a5bea6535b admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:40.309277 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393df39a-383c-48b2-8f64-55a5bea6535b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:40.311100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393df39a-383c-48b2-8f64-55a5bea6535b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:40.327026 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1071/2164] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:40.337215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039fb729-dbcd-4db4-8658-537d3deaeb10 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:40.342468 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039fb729-dbcd-4db4-8658-537d3deaeb10 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:40.347142 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fda77-6ba4-472c-bb55-a87239a7f30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:40.348142 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fda77-6ba4-472c-bb55-a87239a7f30e admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:40.348489 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fda77-6ba4-472c-bb55-a87239a7f30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:40.348836 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fda77-6ba4-472c-bb55-a87239a7f30e admin admin] PATH_INFO: `/v3/projects/78679f3700e04c8fa9ace9086eec127a/users/3201d179294e401b87730ac5dcfa9ce0/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:40.364212 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681fda77-6ba4-472c-bb55-a87239a7f30e admin admin] RBAC: Authorizing `identity:create_grant(project_id=78679f3700e04c8fa9ace9086eec127a, user_id=3201d179294e401b87730ac5dcfa9ce0, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:40.367111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681fda77-6ba4-472c-bb55-a87239a7f30e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:40.386939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1094/2165] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:40 2022] PUT /identity/v3/projects/78679f3700e04c8fa9ace9086eec127a/users/3201d179294e401b87730ac5dcfa9ce0/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:40.401536 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222d490-05b6-4779-abdb-8b518871aa77 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:40.401901 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222d490-05b6-4779-abdb-8b518871aa77 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:40.402212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222d490-05b6-4779-abdb-8b518871aa77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:40.429350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e222d490-05b6-4779-abdb-8b518871aa77 None None] MFA Rules not processed for user `3201d179294e401b87730ac5dcfa9ce0`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:40.430652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e222d490-05b6-4779-abdb-8b518871aa77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:40.501979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1072/2166] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:40.513008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fda77-6ba4-472c-bb55-a87239a7f30e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:40.515682 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-681fda77-6ba4-472c-bb55-a87239a7f30e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:40.549379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:40.579900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fda77-6ba4-472c-bb55-a87239a7f30e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:40.583963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054a499-2a27-4e65-80f7-1a2f0f2f8871 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:40.584617 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054a499-2a27-4e65-80f7-1a2f0f2f8871 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:40.584882 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054a499-2a27-4e65-80f7-1a2f0f2f8871 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:40.585151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054a499-2a27-4e65-80f7-1a2f0f2f8871 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:40.594001 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054a499-2a27-4e65-80f7-1a2f0f2f8871 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:40.595451 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054a499-2a27-4e65-80f7-1a2f0f2f8871 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:40.604559 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1095/2167] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:46.862331 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8bc7b-d457-41b4-b546-c80ca8e8d7c4 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:46.863096 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8bc7b-d457-41b4-b546-c80ca8e8d7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:46.863096 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8bc7b-d457-41b4-b546-c80ca8e8d7c4 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:46.864749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1073/2168] 10.222.0.27 () {54 vars in 921 bytes} [Mon Jun 27 21:17:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:17:46.875314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e1a6f-8738-4ad9-ba71-d4da4ad3b1ef None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:46.877090 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e1a6f-8738-4ad9-ba71-d4da4ad3b1ef None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:46.877542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e1a6f-8738-4ad9-ba71-d4da4ad3b1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:46.896361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b03e1a6f-8738-4ad9-ba71-d4da4ad3b1ef None None] MFA Rules not processed for user `1f5fa9b0b0314ee9bb7e16b8464307b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:46.897584 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b03e1a6f-8738-4ad9-ba71-d4da4ad3b1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:46.990283 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1096/2169] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Jun 27 21:17:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:47.003851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8bc7b-d457-41b4-b546-c80ca8e8d7c4 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:47.005712 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-25f8bc7b-d457-41b4-b546-c80ca8e8d7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:47.032702 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8bc7b-d457-41b4-b546-c80ca8e8d7c4 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:47.045037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a5c44-239e-4538-9509-7eece6e31b7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5fa9b0b0314ee9bb7e16b8464307b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:47.046130 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a5c44-239e-4538-9509-7eece6e31b7a service placement] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:47.046422 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a5c44-239e-4538-9509-7eece6e31b7a service placement] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:47.046705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a5c44-239e-4538-9509-7eece6e31b7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:47.070733 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a5c44-239e-4538-9509-7eece6e31b7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:47.077342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a5c44-239e-4538-9509-7eece6e31b7a service placement] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:47.091661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1074/2170] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:47.491803 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e1a6f-8738-4ad9-ba71-d4da4ad3b1ef None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:47.493353 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b03e1a6f-8738-4ad9-ba71-d4da4ad3b1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:47.562255 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e1a6f-8738-4ad9-ba71-d4da4ad3b1ef None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:47.567545 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a3f0f-c989-4b1d-8a1c-40b07a04abdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:47.568464 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a3f0f-c989-4b1d-8a1c-40b07a04abdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:47.568785 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a3f0f-c989-4b1d-8a1c-40b07a04abdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:47.569039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a3f0f-c989-4b1d-8a1c-40b07a04abdb admin admin] PATH_INFO: `/v3/users/8df15a5fbf8e498ba02552c2e24a3dda` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:47.570276 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a3f0f-c989-4b1d-8a1c-40b07a04abdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8df15a5fbf8e498ba02552c2e24a3dda)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:47.572245 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a3f0f-c989-4b1d-8a1c-40b07a04abdb admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:47.623834 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d65a3f0f-c989-4b1d-8a1c-40b07a04abdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8df15a5fbf8e498ba02552c2e24a3dda'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.624496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d65a3f0f-c989-4b1d-8a1c-40b07a04abdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8df15a5fbf8e498ba02552c2e24a3dda'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.625071 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d65a3f0f-c989-4b1d-8a1c-40b07a04abdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8df15a5fbf8e498ba02552c2e24a3dda'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.638521 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d65a3f0f-c989-4b1d-8a1c-40b07a04abdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8df15a5fbf8e498ba02552c2e24a3dda'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.643990 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:47.647668 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d65a3f0f-c989-4b1d-8a1c-40b07a04abdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8df15a5fbf8e498ba02552c2e24a3dda'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.662736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1097/2171] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:47 2022] DELETE /identity/v3/users/8df15a5fbf8e498ba02552c2e24a3dda => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:47.841559 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a5c44-239e-4538-9509-7eece6e31b7a service placement] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:47.854922 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a5c44-239e-4538-9509-7eece6e31b7a service placement] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:47.913813 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d938dbc-75a5-46e8-878f-0daaca58399b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:47.914683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d938dbc-75a5-46e8-878f-0daaca58399b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:47.914968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d938dbc-75a5-46e8-878f-0daaca58399b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:47.915233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d938dbc-75a5-46e8-878f-0daaca58399b admin admin] PATH_INFO: `/v3/projects/cb85279246c142a98e519de1db70c610` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:47.916449 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d938dbc-75a5-46e8-878f-0daaca58399b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb85279246c142a98e519de1db70c610)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:47.918522 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d938dbc-75a5-46e8-878f-0daaca58399b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:47.924809 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:47.966439 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2d938dbc-75a5-46e8-878f-0daaca58399b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb85279246c142a98e519de1db70c610', 'request_id': 'req-2d938dbc-75a5-46e8-878f-0daaca58399b'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.975106 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2d938dbc-75a5-46e8-878f-0daaca58399b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb85279246c142a98e519de1db70c610', 'request_id': 'req-2d938dbc-75a5-46e8-878f-0daaca58399b'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.980820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2d938dbc-75a5-46e8-878f-0daaca58399b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb85279246c142a98e519de1db70c610', 'request_id': 'req-2d938dbc-75a5-46e8-878f-0daaca58399b'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.992582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2d938dbc-75a5-46e8-878f-0daaca58399b admin admin] The token cache is being invalidate because project cb85279246c142a98e519de1db70c610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:47.992877 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2d938dbc-75a5-46e8-878f-0daaca58399b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.993693 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2d938dbc-75a5-46e8-878f-0daaca58399b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.995322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1075/2172] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:47 2022] DELETE /identity/v3/projects/cb85279246c142a98e519de1db70c610 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:47.999418 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a3f0f-c989-4b1d-8a1c-40b07a04abdb admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:48.001128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d65a3f0f-c989-4b1d-8a1c-40b07a04abdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:48.015591 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a3f0f-c989-4b1d-8a1c-40b07a04abdb admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:48.077125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af8afb-118a-41f0-ae68-076a61cb151a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:48.078017 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af8afb-118a-41f0-ae68-076a61cb151a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.078296 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af8afb-118a-41f0-ae68-076a61cb151a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.078566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af8afb-118a-41f0-ae68-076a61cb151a admin admin] PATH_INFO: `/v3/users/02bcbca6c4e9446c9f00bdf2cd174fad` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.079860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af8afb-118a-41f0-ae68-076a61cb151a admin admin] RBAC: Authorizing `identity:delete_user(user_id=02bcbca6c4e9446c9f00bdf2cd174fad)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:48.081895 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af8afb-118a-41f0-ae68-076a61cb151a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:48.132078 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-92af8afb-118a-41f0-ae68-076a61cb151a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02bcbca6c4e9446c9f00bdf2cd174fad'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:48.132843 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-92af8afb-118a-41f0-ae68-076a61cb151a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02bcbca6c4e9446c9f00bdf2cd174fad'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:48.133489 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-92af8afb-118a-41f0-ae68-076a61cb151a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02bcbca6c4e9446c9f00bdf2cd174fad'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:48.146544 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-92af8afb-118a-41f0-ae68-076a61cb151a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02bcbca6c4e9446c9f00bdf2cd174fad'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:48.153285 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-92af8afb-118a-41f0-ae68-076a61cb151a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02bcbca6c4e9446c9f00bdf2cd174fad'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:48.167104 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1098/2173] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:47 2022] DELETE /identity/v3/users/02bcbca6c4e9446c9f00bdf2cd174fad => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:48.396993 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d938dbc-75a5-46e8-878f-0daaca58399b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:48.398482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2d938dbc-75a5-46e8-878f-0daaca58399b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:48.411993 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d938dbc-75a5-46e8-878f-0daaca58399b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:48.477584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa510c35-2a65-416e-8483-6c7ff052d724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:48.478560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa510c35-2a65-416e-8483-6c7ff052d724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.478846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa510c35-2a65-416e-8483-6c7ff052d724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.479123 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa510c35-2a65-416e-8483-6c7ff052d724 admin admin] PATH_INFO: `/v3/projects/663853349fcc4b0f9c969d4e98300685` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.480373 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa510c35-2a65-416e-8483-6c7ff052d724 admin admin] RBAC: Authorizing `identity:delete_project(project_id=663853349fcc4b0f9c969d4e98300685)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:48.482309 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa510c35-2a65-416e-8483-6c7ff052d724 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:48.530549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fa510c35-2a65-416e-8483-6c7ff052d724 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '663853349fcc4b0f9c969d4e98300685', 'request_id': 'req-fa510c35-2a65-416e-8483-6c7ff052d724'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:48.539848 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fa510c35-2a65-416e-8483-6c7ff052d724 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '663853349fcc4b0f9c969d4e98300685', 'request_id': 'req-fa510c35-2a65-416e-8483-6c7ff052d724'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:48.546123 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fa510c35-2a65-416e-8483-6c7ff052d724 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '663853349fcc4b0f9c969d4e98300685', 'request_id': 'req-fa510c35-2a65-416e-8483-6c7ff052d724'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:48.557630 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fa510c35-2a65-416e-8483-6c7ff052d724 admin admin] The token cache is being invalidate because project 663853349fcc4b0f9c969d4e98300685 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:48.557862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fa510c35-2a65-416e-8483-6c7ff052d724 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:48.558525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fa510c35-2a65-416e-8483-6c7ff052d724 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:48.560170 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1076/2174] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:48 2022] DELETE /identity/v3/projects/663853349fcc4b0f9c969d4e98300685 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:48.565152 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258109ec-7576-4270-a0f0-52aefc38cd0f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.565436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258109ec-7576-4270-a0f0-52aefc38cd0f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.565739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258109ec-7576-4270-a0f0-52aefc38cd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.584336 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-258109ec-7576-4270-a0f0-52aefc38cd0f None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:48.585588 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-258109ec-7576-4270-a0f0-52aefc38cd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:48.647576 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1099/2175] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:48.656294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcd9ac-4f78-41ca-bf10-4fdf88c9b5a0 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.656586 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcd9ac-4f78-41ca-bf10-4fdf88c9b5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.656862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcd9ac-4f78-41ca-bf10-4fdf88c9b5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.676275 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1afcd9ac-4f78-41ca-bf10-4fdf88c9b5a0 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:48.677493 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1afcd9ac-4f78-41ca-bf10-4fdf88c9b5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:48.686228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1077/2176] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:48.690545 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258109ec-7576-4270-a0f0-52aefc38cd0f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:48.701107 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258109ec-7576-4270-a0f0-52aefc38cd0f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:48.706846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707c65d-e60d-4179-b9eb-4bbedd5bd6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:48.707512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707c65d-e60d-4179-b9eb-4bbedd5bd6f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.707859 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707c65d-e60d-4179-b9eb-4bbedd5bd6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.708139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707c65d-e60d-4179-b9eb-4bbedd5bd6f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.708869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c707c65d-e60d-4179-b9eb-4bbedd5bd6f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:48.709181 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c707c65d-e60d-4179-b9eb-4bbedd5bd6f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:48.710805 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c707c65d-e60d-4179-b9eb-4bbedd5bd6f4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:48.712392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:48.726327 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1100/2177] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:17:48.730389 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcd9ac-4f78-41ca-bf10-4fdf88c9b5a0 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:48.740942 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcd9ac-4f78-41ca-bf10-4fdf88c9b5a0 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:48.747588 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41c364-158b-4ca4-91b5-bbd468a70902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:48.748490 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41c364-158b-4ca4-91b5-bbd468a70902 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.748730 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41c364-158b-4ca4-91b5-bbd468a70902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.749011 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41c364-158b-4ca4-91b5-bbd468a70902 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.749712 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41c364-158b-4ca4-91b5-bbd468a70902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:48.751652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41c364-158b-4ca4-91b5-bbd468a70902 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:48.767945 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:48.783546 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1078/2178] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:48 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:48.787803 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707c65d-e60d-4179-b9eb-4bbedd5bd6f4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:48.798159 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707c65d-e60d-4179-b9eb-4bbedd5bd6f4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:48.858892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113bb512-4684-4f3d-9c51-6210068fdf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:48.859712 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bb512-4684-4f3d-9c51-6210068fdf23 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.860052 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bb512-4684-4f3d-9c51-6210068fdf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.860388 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bb512-4684-4f3d-9c51-6210068fdf23 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.861141 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113bb512-4684-4f3d-9c51-6210068fdf23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:48.863136 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113bb512-4684-4f3d-9c51-6210068fdf23 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:48.891644 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1101/2179] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:48 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:17:48.893868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41c364-158b-4ca4-91b5-bbd468a70902 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:48.904977 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41c364-158b-4ca4-91b5-bbd468a70902 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:48.909539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dc661-35da-4f9b-937c-667c987a2f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:48.910199 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dc661-35da-4f9b-937c-667c987a2f8a admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.910483 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dc661-35da-4f9b-937c-667c987a2f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.910755 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dc661-35da-4f9b-937c-667c987a2f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.911479 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588dc661-35da-4f9b-937c-667c987a2f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:48.913099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588dc661-35da-4f9b-937c-667c987a2f8a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:48.923837 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1079/2180] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:48.927848 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113bb512-4684-4f3d-9c51-6210068fdf23 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:48.937796 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113bb512-4684-4f3d-9c51-6210068fdf23 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:48.942308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db13b9-37bd-448a-b3f2-ff10366475fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:48.942962 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db13b9-37bd-448a-b3f2-ff10366475fa admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.943246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db13b9-37bd-448a-b3f2-ff10366475fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.943518 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db13b9-37bd-448a-b3f2-ff10366475fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.944272 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db13b9-37bd-448a-b3f2-ff10366475fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:48.945881 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db13b9-37bd-448a-b3f2-ff10366475fa admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:48.955847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1102/2181] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:48.960055 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dc661-35da-4f9b-937c-667c987a2f8a admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:48.970938 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dc661-35da-4f9b-937c-667c987a2f8a admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:48.976820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba430d-3f96-45f8-840a-79c3fcb4233d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:48.977784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba430d-3f96-45f8-840a-79c3fcb4233d admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.978063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba430d-3f96-45f8-840a-79c3fcb4233d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.978356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba430d-3f96-45f8-840a-79c3fcb4233d admin admin] PATH_INFO: `/v3/projects/60b6a348c84f4aabbe0374589dd9abad/users/74bda19bf0bb42ccb274679eba5be1ea/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.994547 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afba430d-3f96-45f8-840a-79c3fcb4233d admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b6a348c84f4aabbe0374589dd9abad, user_id=74bda19bf0bb42ccb274679eba5be1ea, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:48.997313 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afba430d-3f96-45f8-840a-79c3fcb4233d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:49.013047 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1080/2182] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:48 2022] PUT /identity/v3/projects/60b6a348c84f4aabbe0374589dd9abad/users/74bda19bf0bb42ccb274679eba5be1ea/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:49.028400 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445be42-e783-4423-895e-0c87f9d3e93d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:49.028863 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445be42-e783-4423-895e-0c87f9d3e93d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:49.028952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445be42-e783-4423-895e-0c87f9d3e93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:49.054925 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0445be42-e783-4423-895e-0c87f9d3e93d None None] MFA Rules not processed for user `74bda19bf0bb42ccb274679eba5be1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:49.056152 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0445be42-e783-4423-895e-0c87f9d3e93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:49.120990 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1103/2183] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:49.131762 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba430d-3f96-45f8-840a-79c3fcb4233d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:49.133677 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-afba430d-3f96-45f8-840a-79c3fcb4233d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:49.192272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba430d-3f96-45f8-840a-79c3fcb4233d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:49.196002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d835ec-a696-41f1-a507-b15d886053b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:49.196641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d835ec-a696-41f1-a507-b15d886053b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:49.196920 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d835ec-a696-41f1-a507-b15d886053b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:49.197193 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d835ec-a696-41f1-a507-b15d886053b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:49.205743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d835ec-a696-41f1-a507-b15d886053b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:49.207177 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d835ec-a696-41f1-a507-b15d886053b8 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:49.216102 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1081/2184] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:49.234719 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445be42-e783-4423-895e-0c87f9d3e93d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:49.299676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445be42-e783-4423-895e-0c87f9d3e93d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:49.304691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8e64c-0eb5-4e1b-b3cc-e2ee10a19c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:49.305572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8e64c-0eb5-4e1b-b3cc-e2ee10a19c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:49.305855 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8e64c-0eb5-4e1b-b3cc-e2ee10a19c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:49.306132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8e64c-0eb5-4e1b-b3cc-e2ee10a19c8f admin admin] PATH_INFO: `/v3/users/74bda19bf0bb42ccb274679eba5be1ea` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:49.307402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8e64c-0eb5-4e1b-b3cc-e2ee10a19c8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=74bda19bf0bb42ccb274679eba5be1ea)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:49.309419 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8e64c-0eb5-4e1b-b3cc-e2ee10a19c8f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:49.311896 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:49.358397 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d9d8e64c-0eb5-4e1b-b3cc-e2ee10a19c8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74bda19bf0bb42ccb274679eba5be1ea'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:49.359095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d9d8e64c-0eb5-4e1b-b3cc-e2ee10a19c8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74bda19bf0bb42ccb274679eba5be1ea'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:49.359780 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d9d8e64c-0eb5-4e1b-b3cc-e2ee10a19c8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74bda19bf0bb42ccb274679eba5be1ea'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:49.373098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d9d8e64c-0eb5-4e1b-b3cc-e2ee10a19c8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74bda19bf0bb42ccb274679eba5be1ea'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:49.386137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d9d8e64c-0eb5-4e1b-b3cc-e2ee10a19c8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74bda19bf0bb42ccb274679eba5be1ea'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:49.402868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1104/2185] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:49 2022] DELETE /identity/v3/users/74bda19bf0bb42ccb274679eba5be1ea => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:49.407418 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f905a8-c034-4580-b9a6-12519bd2a823 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:49.407735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f905a8-c034-4580-b9a6-12519bd2a823 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:49.408014 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f905a8-c034-4580-b9a6-12519bd2a823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:49.425233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-03f905a8-c034-4580-b9a6-12519bd2a823 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:49.426433 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-03f905a8-c034-4580-b9a6-12519bd2a823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:49.489051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1082/2186] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:49.500258 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8e64c-0eb5-4e1b-b3cc-e2ee10a19c8f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:49.501964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d9d8e64c-0eb5-4e1b-b3cc-e2ee10a19c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:49.563510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8e64c-0eb5-4e1b-b3cc-e2ee10a19c8f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:49.567571 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dbff2c-fe37-4b77-892c-544f5c81dbf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:49.568267 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbff2c-fe37-4b77-892c-544f5c81dbf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:49.568551 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbff2c-fe37-4b77-892c-544f5c81dbf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:49.568828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbff2c-fe37-4b77-892c-544f5c81dbf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:49.578363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dbff2c-fe37-4b77-892c-544f5c81dbf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:49.579875 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dbff2c-fe37-4b77-892c-544f5c81dbf8 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:49.589560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1105/2187] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:50.241539 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f905a8-c034-4580-b9a6-12519bd2a823 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:50.243178 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-03f905a8-c034-4580-b9a6-12519bd2a823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:50.258498 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f905a8-c034-4580-b9a6-12519bd2a823 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:50.264035 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dd4c3-0766-47f8-b886-c94d1e362ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:50.266580 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dd4c3-0766-47f8-b886-c94d1e362ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:50.267028 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dd4c3-0766-47f8-b886-c94d1e362ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:50.267454 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dd4c3-0766-47f8-b886-c94d1e362ad8 admin admin] PATH_INFO: `/v3/projects/60b6a348c84f4aabbe0374589dd9abad` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:50.270391 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2dd4c3-0766-47f8-b886-c94d1e362ad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b6a348c84f4aabbe0374589dd9abad)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:50.273116 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2dd4c3-0766-47f8-b886-c94d1e362ad8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:50.334120 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fa2dd4c3-0766-47f8-b886-c94d1e362ad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b6a348c84f4aabbe0374589dd9abad', 'request_id': 'req-fa2dd4c3-0766-47f8-b886-c94d1e362ad8'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:50.345721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fa2dd4c3-0766-47f8-b886-c94d1e362ad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b6a348c84f4aabbe0374589dd9abad', 'request_id': 'req-fa2dd4c3-0766-47f8-b886-c94d1e362ad8'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:50.348554 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:50.358139 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fa2dd4c3-0766-47f8-b886-c94d1e362ad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b6a348c84f4aabbe0374589dd9abad', 'request_id': 'req-fa2dd4c3-0766-47f8-b886-c94d1e362ad8'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:50.371912 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fa2dd4c3-0766-47f8-b886-c94d1e362ad8 admin admin] The token cache is being invalidate because project 60b6a348c84f4aabbe0374589dd9abad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:50.372232 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fa2dd4c3-0766-47f8-b886-c94d1e362ad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:50.372922 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fa2dd4c3-0766-47f8-b886-c94d1e362ad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:50.376377 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1083/2188] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:50 2022] DELETE /identity/v3/projects/60b6a348c84f4aabbe0374589dd9abad => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:17:50.381411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d7682-a64b-487e-b3ae-0892603f8e15 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:50.381888 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d7682-a64b-487e-b3ae-0892603f8e15 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:50.381981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d7682-a64b-487e-b3ae-0892603f8e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:50.403829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c51d7682-a64b-487e-b3ae-0892603f8e15 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:50.405079 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c51d7682-a64b-487e-b3ae-0892603f8e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:50.471343 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1106/2189] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:50.479582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711c199-baf0-4683-9f69-874dd96f383c None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:50.479942 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711c199-baf0-4683-9f69-874dd96f383c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:50.480238 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711c199-baf0-4683-9f69-874dd96f383c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:50.502648 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b711c199-baf0-4683-9f69-874dd96f383c None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:50.503913 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b711c199-baf0-4683-9f69-874dd96f383c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:50.512791 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1084/2190] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:50.517317 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d7682-a64b-487e-b3ae-0892603f8e15 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:50.528197 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d7682-a64b-487e-b3ae-0892603f8e15 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:50.533147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f60e6-8390-47c9-b66f-65c7569511a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:50.534858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f60e6-8390-47c9-b66f-65c7569511a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:50.535148 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f60e6-8390-47c9-b66f-65c7569511a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:50.535448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f60e6-8390-47c9-b66f-65c7569511a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:50.536228 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f60e6-8390-47c9-b66f-65c7569511a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:50.536546 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f60e6-8390-47c9-b66f-65c7569511a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:50.538163 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f60e6-8390-47c9-b66f-65c7569511a4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:50.551631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1107/2191] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:17:50.554598 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711c199-baf0-4683-9f69-874dd96f383c None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:50.564568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711c199-baf0-4683-9f69-874dd96f383c None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:50.570299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17760075-965c-4424-96f3-09bee358674c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:50.570960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17760075-965c-4424-96f3-09bee358674c admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:50.571244 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17760075-965c-4424-96f3-09bee358674c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:50.571515 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17760075-965c-4424-96f3-09bee358674c admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:50.572245 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17760075-965c-4424-96f3-09bee358674c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:50.574166 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17760075-965c-4424-96f3-09bee358674c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:50.604220 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1085/2192] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:50 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:50.605443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f60e6-8390-47c9-b66f-65c7569511a4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:50.615862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f60e6-8390-47c9-b66f-65c7569511a4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:50.626246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:50.697799 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5acee3-bb6b-4e16-a329-fec6242856b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:50.699342 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5acee3-bb6b-4e16-a329-fec6242856b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:50.699342 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5acee3-bb6b-4e16-a329-fec6242856b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:50.699631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5acee3-bb6b-4e16-a329-fec6242856b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:50.701223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5acee3-bb6b-4e16-a329-fec6242856b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:50.702765 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5acee3-bb6b-4e16-a329-fec6242856b3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:50.733551 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1108/2193] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:50 2022] POST /identity/v3/users => generated 286 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:17:50.737756 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17760075-965c-4424-96f3-09bee358674c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:50.750403 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17760075-965c-4424-96f3-09bee358674c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:50.755193 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23798cf1-00fa-451f-b451-b79824b57412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:50.755905 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23798cf1-00fa-451f-b451-b79824b57412 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:50.756190 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23798cf1-00fa-451f-b451-b79824b57412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:50.756469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23798cf1-00fa-451f-b451-b79824b57412 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:50.757226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23798cf1-00fa-451f-b451-b79824b57412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:50.758912 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23798cf1-00fa-451f-b451-b79824b57412 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:50.772073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1086/2194] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:50.776248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5acee3-bb6b-4e16-a329-fec6242856b3 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:50.786795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5acee3-bb6b-4e16-a329-fec6242856b3 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:50.793075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82878f-9247-44d9-83ab-52ec04e13ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:50.793770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82878f-9247-44d9-83ab-52ec04e13ebf admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:50.794387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82878f-9247-44d9-83ab-52ec04e13ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:50.794899 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82878f-9247-44d9-83ab-52ec04e13ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:50.796320 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a82878f-9247-44d9-83ab-52ec04e13ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:50.798702 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a82878f-9247-44d9-83ab-52ec04e13ebf admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:50.812091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1109/2195] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:50.816526 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23798cf1-00fa-451f-b451-b79824b57412 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:50.826784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23798cf1-00fa-451f-b451-b79824b57412 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:50.831734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b55d9-575b-4113-bc87-a01f9c6ec598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:50.832622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b55d9-575b-4113-bc87-a01f9c6ec598 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:50.832906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b55d9-575b-4113-bc87-a01f9c6ec598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:50.833747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b55d9-575b-4113-bc87-a01f9c6ec598 admin admin] PATH_INFO: `/v3/projects/50e36bac806e4c49a8a2eb89f3566afa/users/b21235c91fb143359775b346f6234c9f/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:50.849358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b55d9-575b-4113-bc87-a01f9c6ec598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50e36bac806e4c49a8a2eb89f3566afa, user_id=b21235c91fb143359775b346f6234c9f, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:50.852150 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b55d9-575b-4113-bc87-a01f9c6ec598 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:50.868282 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1087/2196] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:50 2022] PUT /identity/v3/projects/50e36bac806e4c49a8a2eb89f3566afa/users/b21235c91fb143359775b346f6234c9f/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:17:50.879426 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afdb706-3bd9-4e1a-a9d6-b073839d3482 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:50.879811 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afdb706-3bd9-4e1a-a9d6-b073839d3482 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:50.880088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afdb706-3bd9-4e1a-a9d6-b073839d3482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:50.907083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6afdb706-3bd9-4e1a-a9d6-b073839d3482 None None] MFA Rules not processed for user `b21235c91fb143359775b346f6234c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:50.908416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6afdb706-3bd9-4e1a-a9d6-b073839d3482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:50.980006 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1110/2197] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:50.991450 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b55d9-575b-4113-bc87-a01f9c6ec598 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:50.993345 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fd9b55d9-575b-4113-bc87-a01f9c6ec598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:51.050776 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b55d9-575b-4113-bc87-a01f9c6ec598 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:51.054547 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b61833-8705-48c0-a902-67ac574098c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:51.055220 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b61833-8705-48c0-a902-67ac574098c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:51.055502 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b61833-8705-48c0-a902-67ac574098c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:51.055817 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b61833-8705-48c0-a902-67ac574098c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:51.066475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b61833-8705-48c0-a902-67ac574098c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:51.068004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b61833-8705-48c0-a902-67ac574098c2 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:51.081480 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1088/2198] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:51.100904 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afdb706-3bd9-4e1a-a9d6-b073839d3482 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:51.167102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afdb706-3bd9-4e1a-a9d6-b073839d3482 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:51.172605 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7291d6c-fab9-4bcb-a041-148ae3de3654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:51.173490 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7291d6c-fab9-4bcb-a041-148ae3de3654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:51.173764 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7291d6c-fab9-4bcb-a041-148ae3de3654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:51.174018 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7291d6c-fab9-4bcb-a041-148ae3de3654 admin admin] PATH_INFO: `/v3/users/b21235c91fb143359775b346f6234c9f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:51.175256 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7291d6c-fab9-4bcb-a041-148ae3de3654 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b21235c91fb143359775b346f6234c9f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:51.177223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7291d6c-fab9-4bcb-a041-148ae3de3654 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:51.202028 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:51.231056 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a7291d6c-fab9-4bcb-a041-148ae3de3654 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b21235c91fb143359775b346f6234c9f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:51.231805 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a7291d6c-fab9-4bcb-a041-148ae3de3654 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b21235c91fb143359775b346f6234c9f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:51.232459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a7291d6c-fab9-4bcb-a041-148ae3de3654 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b21235c91fb143359775b346f6234c9f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:51.244042 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a7291d6c-fab9-4bcb-a041-148ae3de3654 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b21235c91fb143359775b346f6234c9f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:51.251832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a7291d6c-fab9-4bcb-a041-148ae3de3654 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b21235c91fb143359775b346f6234c9f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:51.267548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1111/2199] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:51 2022] DELETE /identity/v3/users/b21235c91fb143359775b346f6234c9f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:51.272179 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb26e19-ff6a-4338-b31e-f05a92698b57 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:51.272678 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb26e19-ff6a-4338-b31e-f05a92698b57 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:51.272768 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb26e19-ff6a-4338-b31e-f05a92698b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:51.292920 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-efb26e19-ff6a-4338-b31e-f05a92698b57 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:51.294152 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-efb26e19-ff6a-4338-b31e-f05a92698b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:51.358842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1089/2200] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:51.368939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7291d6c-fab9-4bcb-a041-148ae3de3654 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:51.371011 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a7291d6c-fab9-4bcb-a041-148ae3de3654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:51.442180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7291d6c-fab9-4bcb-a041-148ae3de3654 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:51.445892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8919b54-20a2-4614-a013-4c869a608809 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:51.446601 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8919b54-20a2-4614-a013-4c869a608809 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:51.446883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8919b54-20a2-4614-a013-4c869a608809 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:51.447218 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8919b54-20a2-4614-a013-4c869a608809 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:51.457109 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8919b54-20a2-4614-a013-4c869a608809 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:51.458558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8919b54-20a2-4614-a013-4c869a608809 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:51.471154 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1112/2201] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:51.774055 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb26e19-ff6a-4338-b31e-f05a92698b57 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:51.775709 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-efb26e19-ff6a-4338-b31e-f05a92698b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:51.789943 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb26e19-ff6a-4338-b31e-f05a92698b57 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:51.807401 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75be9f-6005-4984-9cc1-1978e9cb4d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:51.809515 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75be9f-6005-4984-9cc1-1978e9cb4d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:51.809967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75be9f-6005-4984-9cc1-1978e9cb4d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:51.811268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75be9f-6005-4984-9cc1-1978e9cb4d01 admin admin] PATH_INFO: `/v3/projects/50e36bac806e4c49a8a2eb89f3566afa` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:51.812636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75be9f-6005-4984-9cc1-1978e9cb4d01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50e36bac806e4c49a8a2eb89f3566afa)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:51.814642 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75be9f-6005-4984-9cc1-1978e9cb4d01 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:51.838932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:51.865098 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bb75be9f-6005-4984-9cc1-1978e9cb4d01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50e36bac806e4c49a8a2eb89f3566afa', 'request_id': 'req-bb75be9f-6005-4984-9cc1-1978e9cb4d01'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:51.874912 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bb75be9f-6005-4984-9cc1-1978e9cb4d01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50e36bac806e4c49a8a2eb89f3566afa', 'request_id': 'req-bb75be9f-6005-4984-9cc1-1978e9cb4d01'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:51.882287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bb75be9f-6005-4984-9cc1-1978e9cb4d01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50e36bac806e4c49a8a2eb89f3566afa', 'request_id': 'req-bb75be9f-6005-4984-9cc1-1978e9cb4d01'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:51.894555 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bb75be9f-6005-4984-9cc1-1978e9cb4d01 admin admin] The token cache is being invalidate because project 50e36bac806e4c49a8a2eb89f3566afa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:51.894919 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bb75be9f-6005-4984-9cc1-1978e9cb4d01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:51.895753 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bb75be9f-6005-4984-9cc1-1978e9cb4d01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:51.897370 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1090/2202] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:51 2022] DELETE /identity/v3/projects/50e36bac806e4c49a8a2eb89f3566afa => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:51.902460 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d433585-8430-453c-a0fb-10c7515bf0af None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:51.902748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d433585-8430-453c-a0fb-10c7515bf0af None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:51.903022 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d433585-8430-453c-a0fb-10c7515bf0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:51.920395 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2d433585-8430-453c-a0fb-10c7515bf0af None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:51.921625 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2d433585-8430-453c-a0fb-10c7515bf0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:51.999889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1113/2203] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:52.009264 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f3b39-8290-46b0-be4b-48e947898abd None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.009738 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f3b39-8290-46b0-be4b-48e947898abd None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.010198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f3b39-8290-46b0-be4b-48e947898abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.050074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1b9f3b39-8290-46b0-be4b-48e947898abd None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:52.054285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1b9f3b39-8290-46b0-be4b-48e947898abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:52.063412 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1091/2204] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:52.067861 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d433585-8430-453c-a0fb-10c7515bf0af None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:52.078312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d433585-8430-453c-a0fb-10c7515bf0af None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:52.082895 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad67d9-2389-4b8d-9246-28ece0f954a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:52.083558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad67d9-2389-4b8d-9246-28ece0f954a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.083883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad67d9-2389-4b8d-9246-28ece0f954a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.084149 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad67d9-2389-4b8d-9246-28ece0f954a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.084907 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad67d9-2389-4b8d-9246-28ece0f954a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:52.085214 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad67d9-2389-4b8d-9246-28ece0f954a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:52.086832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad67d9-2389-4b8d-9246-28ece0f954a9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:52.098499 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1114/2205] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:17:52.102679 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f3b39-8290-46b0-be4b-48e947898abd None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:52.111971 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f3b39-8290-46b0-be4b-48e947898abd None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:52.116878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9477f-1fdb-4fed-acab-e42e7cbcce37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:52.117566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9477f-1fdb-4fed-acab-e42e7cbcce37 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.117844 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9477f-1fdb-4fed-acab-e42e7cbcce37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.118145 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9477f-1fdb-4fed-acab-e42e7cbcce37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.118845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9477f-1fdb-4fed-acab-e42e7cbcce37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:52.121015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9477f-1fdb-4fed-acab-e42e7cbcce37 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:52.149237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1092/2206] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:52 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:52.153714 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad67d9-2389-4b8d-9246-28ece0f954a9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:52.164139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad67d9-2389-4b8d-9246-28ece0f954a9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:52.190896 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:52.224567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21975129-9d43-47a6-8296-205eb10da921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:52.225247 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21975129-9d43-47a6-8296-205eb10da921 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.225530 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21975129-9d43-47a6-8296-205eb10da921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.225803 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21975129-9d43-47a6-8296-205eb10da921 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.226506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21975129-9d43-47a6-8296-205eb10da921 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:52.228487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21975129-9d43-47a6-8296-205eb10da921 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:52.247593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1115/2207] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:52 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:52.251851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9477f-1fdb-4fed-acab-e42e7cbcce37 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:52.262529 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9477f-1fdb-4fed-acab-e42e7cbcce37 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:52.268089 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ffd52b-1e0a-4b64-b9ca-7ae12394edc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:52.268785 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffd52b-1e0a-4b64-b9ca-7ae12394edc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.269079 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffd52b-1e0a-4b64-b9ca-7ae12394edc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.269347 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffd52b-1e0a-4b64-b9ca-7ae12394edc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.270082 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ffd52b-1e0a-4b64-b9ca-7ae12394edc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:52.271723 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ffd52b-1e0a-4b64-b9ca-7ae12394edc7 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:52.282906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1093/2208] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:52.286950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21975129-9d43-47a6-8296-205eb10da921 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:52.299653 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21975129-9d43-47a6-8296-205eb10da921 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:52.304653 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd88df21-55b4-4ccd-a018-4a8f3efeb61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:52.305482 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88df21-55b4-4ccd-a018-4a8f3efeb61f admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.305769 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88df21-55b4-4ccd-a018-4a8f3efeb61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.306018 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88df21-55b4-4ccd-a018-4a8f3efeb61f admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.306754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd88df21-55b4-4ccd-a018-4a8f3efeb61f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:52.308533 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd88df21-55b4-4ccd-a018-4a8f3efeb61f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:52.321830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1116/2209] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:52.326193 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ffd52b-1e0a-4b64-b9ca-7ae12394edc7 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:52.336945 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ffd52b-1e0a-4b64-b9ca-7ae12394edc7 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:52.341834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82339bdf-be89-4a78-9d15-29d0023a4e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:52.342732 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82339bdf-be89-4a78-9d15-29d0023a4e2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.343004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82339bdf-be89-4a78-9d15-29d0023a4e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.343282 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82339bdf-be89-4a78-9d15-29d0023a4e2f admin admin] PATH_INFO: `/v3/projects/3410c5719ac1452a8ead4b5f6f0f536f/users/a50fc08b8e7940e3a88248945e1b5f12/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.360508 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82339bdf-be89-4a78-9d15-29d0023a4e2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3410c5719ac1452a8ead4b5f6f0f536f, user_id=a50fc08b8e7940e3a88248945e1b5f12, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:52.363220 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82339bdf-be89-4a78-9d15-29d0023a4e2f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:52.378504 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1094/2210] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:52 2022] PUT /identity/v3/projects/3410c5719ac1452a8ead4b5f6f0f536f/users/a50fc08b8e7940e3a88248945e1b5f12/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:52.388948 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca02bc-295f-45f8-9cd7-aeb6233ecfa1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.389234 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca02bc-295f-45f8-9cd7-aeb6233ecfa1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.389630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca02bc-295f-45f8-9cd7-aeb6233ecfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.417421 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f6ca02bc-295f-45f8-9cd7-aeb6233ecfa1 None None] MFA Rules not processed for user `a50fc08b8e7940e3a88248945e1b5f12`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:52.418623 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f6ca02bc-295f-45f8-9cd7-aeb6233ecfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:52.488684 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1117/2211] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:52.490110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82339bdf-be89-4a78-9d15-29d0023a4e2f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:52.557588 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82339bdf-be89-4a78-9d15-29d0023a4e2f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:52.562477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce85b2-578c-40ff-b18d-1c5869215534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:52.563145 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce85b2-578c-40ff-b18d-1c5869215534 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.563883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce85b2-578c-40ff-b18d-1c5869215534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.563883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce85b2-578c-40ff-b18d-1c5869215534 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.564402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce85b2-578c-40ff-b18d-1c5869215534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:52.566340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce85b2-578c-40ff-b18d-1c5869215534 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:52.599336 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1095/2212] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:52 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:52.603945 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca02bc-295f-45f8-9cd7-aeb6233ecfa1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:52.620991 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca02bc-295f-45f8-9cd7-aeb6233ecfa1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:52.678339 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7513e-0559-452a-b8f5-117752522493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:52.679018 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7513e-0559-452a-b8f5-117752522493 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.679298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7513e-0559-452a-b8f5-117752522493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.679572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7513e-0559-452a-b8f5-117752522493 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.680310 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e7513e-0559-452a-b8f5-117752522493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:52.682300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e7513e-0559-452a-b8f5-117752522493 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:52.707449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1118/2213] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:52 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:17:52.711504 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce85b2-578c-40ff-b18d-1c5869215534 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:52.722164 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce85b2-578c-40ff-b18d-1c5869215534 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:52.727261 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863917b5-2c7f-4282-a515-ea652c051615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:52.727967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863917b5-2c7f-4282-a515-ea652c051615 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.728252 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863917b5-2c7f-4282-a515-ea652c051615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.728558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863917b5-2c7f-4282-a515-ea652c051615 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.729260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863917b5-2c7f-4282-a515-ea652c051615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:52.730895 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863917b5-2c7f-4282-a515-ea652c051615 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:52.741659 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1096/2214] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:52.745690 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7513e-0559-452a-b8f5-117752522493 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:52.756136 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7513e-0559-452a-b8f5-117752522493 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:52.761283 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf360774-194b-4043-aec8-0fa5da277864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:52.761947 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf360774-194b-4043-aec8-0fa5da277864 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.762229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf360774-194b-4043-aec8-0fa5da277864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.762506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf360774-194b-4043-aec8-0fa5da277864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.763321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf360774-194b-4043-aec8-0fa5da277864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:52.764918 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf360774-194b-4043-aec8-0fa5da277864 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:52.776260 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1119/2215] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:52.780286 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863917b5-2c7f-4282-a515-ea652c051615 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:52.790761 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863917b5-2c7f-4282-a515-ea652c051615 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:52.795687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0f141-9311-4de7-832b-095a8cb17d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:52.796604 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0f141-9311-4de7-832b-095a8cb17d3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.797036 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0f141-9311-4de7-832b-095a8cb17d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.797383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0f141-9311-4de7-832b-095a8cb17d3d admin admin] PATH_INFO: `/v3/projects/9897ef22d42e4eba915df5a1f2a5ab13/users/6206f0ddf81b402fb4e3e325c984ec86/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.819877 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0f141-9311-4de7-832b-095a8cb17d3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9897ef22d42e4eba915df5a1f2a5ab13, user_id=6206f0ddf81b402fb4e3e325c984ec86, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:52.822559 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0f141-9311-4de7-832b-095a8cb17d3d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:52.825716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:52.838722 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1097/2216] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:52 2022] PUT /identity/v3/projects/9897ef22d42e4eba915df5a1f2a5ab13/users/6206f0ddf81b402fb4e3e325c984ec86/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:52.851380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80528501-aec6-42e2-bdd6-75baa1465a9d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.851699 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80528501-aec6-42e2-bdd6-75baa1465a9d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.851971 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80528501-aec6-42e2-bdd6-75baa1465a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.882346 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-80528501-aec6-42e2-bdd6-75baa1465a9d None None] MFA Rules not processed for user `6206f0ddf81b402fb4e3e325c984ec86`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:52.883673 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-80528501-aec6-42e2-bdd6-75baa1465a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:52.887915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:52.963577 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1120/2217] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:52.967967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0f141-9311-4de7-832b-095a8cb17d3d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:53.032920 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0f141-9311-4de7-832b-095a8cb17d3d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:53.038231 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923434a1-934c-42f4-b624-42b8196d6a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:53.038882 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923434a1-934c-42f4-b624-42b8196d6a0c admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:53.039166 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923434a1-934c-42f4-b624-42b8196d6a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:53.039564 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923434a1-934c-42f4-b624-42b8196d6a0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:53.040300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923434a1-934c-42f4-b624-42b8196d6a0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:53.042244 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923434a1-934c-42f4-b624-42b8196d6a0c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:53.073585 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1098/2218] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:52 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:53.078270 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80528501-aec6-42e2-bdd6-75baa1465a9d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:53.092990 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80528501-aec6-42e2-bdd6-75baa1465a9d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:53.155222 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40cf8d-0fa8-4493-9959-6df1ed970b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:53.155961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40cf8d-0fa8-4493-9959-6df1ed970b87 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:53.156235 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40cf8d-0fa8-4493-9959-6df1ed970b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:53.156508 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40cf8d-0fa8-4493-9959-6df1ed970b87 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:53.157216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40cf8d-0fa8-4493-9959-6df1ed970b87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:53.159161 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40cf8d-0fa8-4493-9959-6df1ed970b87 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:53.180778 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1121/2219] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:53 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:53.186199 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923434a1-934c-42f4-b624-42b8196d6a0c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:53.201490 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923434a1-934c-42f4-b624-42b8196d6a0c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:53.209315 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f7e6c-38c7-4994-8aaa-1900a2a368d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:53.210030 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f7e6c-38c7-4994-8aaa-1900a2a368d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:53.210317 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f7e6c-38c7-4994-8aaa-1900a2a368d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:53.210622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f7e6c-38c7-4994-8aaa-1900a2a368d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:53.211351 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f7e6c-38c7-4994-8aaa-1900a2a368d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:53.213090 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f7e6c-38c7-4994-8aaa-1900a2a368d3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:53.225190 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1099/2220] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:17:53.229357 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40cf8d-0fa8-4493-9959-6df1ed970b87 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:53.239768 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40cf8d-0fa8-4493-9959-6df1ed970b87 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:53.244421 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9d1a9-e166-403b-9d1d-722c9e3df71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:53.245344 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9d1a9-e166-403b-9d1d-722c9e3df71e admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:53.245627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9d1a9-e166-403b-9d1d-722c9e3df71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:53.245905 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9d1a9-e166-403b-9d1d-722c9e3df71e admin admin] PATH_INFO: `/v3/projects/ce41f01380424924bf480db36f7a4650/users/53093b67f7e145539b18ef59e3e4e21b/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:53.261038 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9d1a9-e166-403b-9d1d-722c9e3df71e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce41f01380424924bf480db36f7a4650, user_id=53093b67f7e145539b18ef59e3e4e21b, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:53.263782 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9d1a9-e166-403b-9d1d-722c9e3df71e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:53.279086 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1122/2221] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:53 2022] PUT /identity/v3/projects/ce41f01380424924bf480db36f7a4650/users/53093b67f7e145539b18ef59e3e4e21b/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:53.283098 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f7e6c-38c7-4994-8aaa-1900a2a368d3 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:53.293220 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f7e6c-38c7-4994-8aaa-1900a2a368d3 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:53.350532 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a63ea-d298-460b-8b4f-0d10265f3d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:53.351194 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a63ea-d298-460b-8b4f-0d10265f3d73 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:53.351474 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a63ea-d298-460b-8b4f-0d10265f3d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:53.351784 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a63ea-d298-460b-8b4f-0d10265f3d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:53.352508 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a63ea-d298-460b-8b4f-0d10265f3d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:53.354109 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a63ea-d298-460b-8b4f-0d10265f3d73 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:53.364005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1100/2222] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:53.368342 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9d1a9-e166-403b-9d1d-722c9e3df71e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:53.379203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9d1a9-e166-403b-9d1d-722c9e3df71e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:53.384650 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd11f48-160e-4576-9d79-aadd1b48bc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:53.385632 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd11f48-160e-4576-9d79-aadd1b48bc52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:53.385940 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd11f48-160e-4576-9d79-aadd1b48bc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:53.386279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd11f48-160e-4576-9d79-aadd1b48bc52 admin admin] PATH_INFO: `/v3/projects/ce41f01380424924bf480db36f7a4650/users/53093b67f7e145539b18ef59e3e4e21b/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:53.388350 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd11f48-160e-4576-9d79-aadd1b48bc52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce41f01380424924bf480db36f7a4650, user_id=53093b67f7e145539b18ef59e3e4e21b, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:53.391347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd11f48-160e-4576-9d79-aadd1b48bc52 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:53.402487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1123/2223] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:53 2022] PUT /identity/v3/projects/ce41f01380424924bf480db36f7a4650/users/53093b67f7e145539b18ef59e3e4e21b/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:53.413845 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858df28c-2101-48c6-88b4-3608f991eb4f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:53.414197 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858df28c-2101-48c6-88b4-3608f991eb4f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:53.414565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858df28c-2101-48c6-88b4-3608f991eb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:53.448909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-858df28c-2101-48c6-88b4-3608f991eb4f None None] MFA Rules not processed for user `53093b67f7e145539b18ef59e3e4e21b`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:53.450948 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-858df28c-2101-48c6-88b4-3608f991eb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:53.511628 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:53.531746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1101/2224] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:17:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:53.544186 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd11f48-160e-4576-9d79-aadd1b48bc52 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:53.546420 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5cd11f48-160e-4576-9d79-aadd1b48bc52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:53.609700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd11f48-160e-4576-9d79-aadd1b48bc52 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:53.613753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3929e-4ccb-48ce-9a5d-b78fc1ce55e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:53.614406 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3929e-4ccb-48ce-9a5d-b78fc1ce55e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:53.614689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3929e-4ccb-48ce-9a5d-b78fc1ce55e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:53.614967 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3929e-4ccb-48ce-9a5d-b78fc1ce55e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:53.624875 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:53.677123 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3929e-4ccb-48ce-9a5d-b78fc1ce55e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:53.678727 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3929e-4ccb-48ce-9a5d-b78fc1ce55e2 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:53.688160 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1124/2225] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:55.222246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858df28c-2101-48c6-88b4-3608f991eb4f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:55.232270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858df28c-2101-48c6-88b4-3608f991eb4f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:55.236084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2493ab6-5f70-4d28-81c2-b04b890677db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:55.236721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2493ab6-5f70-4d28-81c2-b04b890677db service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:55.237000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2493ab6-5f70-4d28-81c2-b04b890677db service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:55.237276 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2493ab6-5f70-4d28-81c2-b04b890677db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:55.321629 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2493ab6-5f70-4d28-81c2-b04b890677db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:55.324191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2493ab6-5f70-4d28-81c2-b04b890677db service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:55.333821 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1102/2226] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:56.024286 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3929e-4ccb-48ce-9a5d-b78fc1ce55e2 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:56.032941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3929e-4ccb-48ce-9a5d-b78fc1ce55e2 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:56.036915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf63751-0675-48ee-abec-b4dc845b91c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:56.037558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf63751-0675-48ee-abec-b4dc845b91c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:56.037810 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf63751-0675-48ee-abec-b4dc845b91c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:56.038115 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf63751-0675-48ee-abec-b4dc845b91c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:56.047358 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf63751-0675-48ee-abec-b4dc845b91c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:56.048823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf63751-0675-48ee-abec-b4dc845b91c1 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:56.058510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1125/2227] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:59.400275 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2484ec-966f-4c55-8c11-937e5d2bd756 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:59.400814 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2484ec-966f-4c55-8c11-937e5d2bd756 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:59.400941 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2484ec-966f-4c55-8c11-937e5d2bd756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:59.418660 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-0a2484ec-966f-4c55-8c11-937e5d2bd756 None None] MFA Rules not processed for user `e802da0b078e457da466a9a7b0f15085`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:59.419885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0a2484ec-966f-4c55-8c11-937e5d2bd756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:59.480967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1103/2228] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Jun 27 21:17:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:59.522581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf63751-0675-48ee-abec-b4dc845b91c1 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:59.524489 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-edf63751-0675-48ee-abec-b4dc845b91c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:59.604065 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf63751-0675-48ee-abec-b4dc845b91c1 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:59.609537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b3a6b-45ea-4e1f-ade9-8c1876337c20 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28acb0322114aebafcf62bdf9a346df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:59.610231 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b3a6b-45ea-4e1f-ade9-8c1876337c20 service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:59.610512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b3a6b-45ea-4e1f-ade9-8c1876337c20 service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:59.610788 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b3a6b-45ea-4e1f-ade9-8c1876337c20 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:59.619814 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b3a6b-45ea-4e1f-ade9-8c1876337c20 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:59.621821 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b3a6b-45ea-4e1f-ade9-8c1876337c20 service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:59.630735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1126/2229] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Jun 27 21:17:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:17:59.948918 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2484ec-966f-4c55-8c11-937e5d2bd756 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:00.003071 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b3a6b-45ea-4e1f-ade9-8c1876337c20 service swift] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:00.004787 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0a5b3a6b-45ea-4e1f-ade9-8c1876337c20 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:00.026740 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2484ec-966f-4c55-8c11-937e5d2bd756 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:00.031405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65bbc9-350e-403c-afb1-b0782656aa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:00.032319 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65bbc9-350e-403c-afb1-b0782656aa7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:00.032607 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65bbc9-350e-403c-afb1-b0782656aa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:00.032884 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65bbc9-350e-403c-afb1-b0782656aa7b admin admin] PATH_INFO: `/v3/users/a50fc08b8e7940e3a88248945e1b5f12` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:00.034109 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a65bbc9-350e-403c-afb1-b0782656aa7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a50fc08b8e7940e3a88248945e1b5f12)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:00.036636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a65bbc9-350e-403c-afb1-b0782656aa7b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:00.092269 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:00.097373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b3a6b-45ea-4e1f-ade9-8c1876337c20 service swift] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:00.105678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d8205-f550-424e-b66b-f8b123fea274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:00.106949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d8205-f550-424e-b66b-f8b123fea274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:00.107324 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d8205-f550-424e-b66b-f8b123fea274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:00.107806 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d8205-f550-424e-b66b-f8b123fea274 admin admin] PATH_INFO: `/v3/users/3201d179294e401b87730ac5dcfa9ce0` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:00.109313 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3d8205-f550-424e-b66b-f8b123fea274 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3201d179294e401b87730ac5dcfa9ce0)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:00.111675 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3d8205-f550-424e-b66b-f8b123fea274 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:00.117310 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9a65bbc9-350e-403c-afb1-b0782656aa7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a50fc08b8e7940e3a88248945e1b5f12'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:00.120971 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9a65bbc9-350e-403c-afb1-b0782656aa7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a50fc08b8e7940e3a88248945e1b5f12'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:00.121621 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9a65bbc9-350e-403c-afb1-b0782656aa7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a50fc08b8e7940e3a88248945e1b5f12'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:00.153537 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9a65bbc9-350e-403c-afb1-b0782656aa7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a50fc08b8e7940e3a88248945e1b5f12'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:00.161786 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:00.164397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9a65bbc9-350e-403c-afb1-b0782656aa7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a50fc08b8e7940e3a88248945e1b5f12'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:00.189067 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1a3d8205-f550-424e-b66b-f8b123fea274 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3201d179294e401b87730ac5dcfa9ce0'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:00.190347 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1a3d8205-f550-424e-b66b-f8b123fea274 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3201d179294e401b87730ac5dcfa9ce0'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:00.191172 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1a3d8205-f550-424e-b66b-f8b123fea274 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3201d179294e401b87730ac5dcfa9ce0'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:00.194194 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1104/2230] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:59 2022] DELETE /identity/v3/users/a50fc08b8e7940e3a88248945e1b5f12 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:00.198290 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ccf06-63e5-4015-8d59-d7cf9f1cff9a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:00.198576 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ccf06-63e5-4015-8d59-d7cf9f1cff9a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:00.198849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ccf06-63e5-4015-8d59-d7cf9f1cff9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:00.209438 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1a3d8205-f550-424e-b66b-f8b123fea274 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3201d179294e401b87730ac5dcfa9ce0'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:00.217989 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1a3d8205-f550-424e-b66b-f8b123fea274 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3201d179294e401b87730ac5dcfa9ce0'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:00.218216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-3d7ccf06-63e5-4015-8d59-d7cf9f1cff9a None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:00.219625 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3d7ccf06-63e5-4015-8d59-d7cf9f1cff9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:00.233871 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1127/2231] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:18:00 2022] DELETE /identity/v3/users/3201d179294e401b87730ac5dcfa9ce0 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:00.238278 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346d469-5add-40b1-b393-fb08ffd6add3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:00.238569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346d469-5add-40b1-b393-fb08ffd6add3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:00.238941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346d469-5add-40b1-b393-fb08ffd6add3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:00.260335 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5346d469-5add-40b1-b393-fb08ffd6add3 None None] MFA Rules not processed for user `e802da0b078e457da466a9a7b0f15085`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:00.261563 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5346d469-5add-40b1-b393-fb08ffd6add3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:00.292024 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1105/2232] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:18:00.293505 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16dd3d-25f4-456a-aebe-9f7535492abb None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:00.293867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16dd3d-25f4-456a-aebe-9f7535492abb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:00.294212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16dd3d-25f4-456a-aebe-9f7535492abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:00.313171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-cc16dd3d-25f4-456a-aebe-9f7535492abb None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:00.314394 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-cc16dd3d-25f4-456a-aebe-9f7535492abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:00.320087 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1128/2233] 10.222.0.27 () {58 vars in 967 bytes} [Mon Jun 27 21:18:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:00.320917 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346d469-5add-40b1-b393-fb08ffd6add3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:00.322282 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5346d469-5add-40b1-b393-fb08ffd6add3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:00.378814 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346d469-5add-40b1-b393-fb08ffd6add3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:00.379007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1106/2234] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:00.379385 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16dd3d-25f4-456a-aebe-9f7535492abb None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:00.380683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-cc16dd3d-25f4-456a-aebe-9f7535492abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:00.385135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d0fc0-e5dc-48b8-afb9-b31bb00a5117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:00.385795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d0fc0-e5dc-48b8-afb9-b31bb00a5117 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:00.386079 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d0fc0-e5dc-48b8-afb9-b31bb00a5117 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:00.386400 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d0fc0-e5dc-48b8-afb9-b31bb00a5117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:00.397014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d0fc0-e5dc-48b8-afb9-b31bb00a5117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:00.398461 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d0fc0-e5dc-48b8-afb9-b31bb00a5117 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:00.409634 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1129/2235] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:00.415560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d0fc0-e5dc-48b8-afb9-b31bb00a5117 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:00.436589 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d0fc0-e5dc-48b8-afb9-b31bb00a5117 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:00.447747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71950a4e-91db-4789-b686-8165f3623c48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:00.448430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71950a4e-91db-4789-b686-8165f3623c48 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:00.448683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71950a4e-91db-4789-b686-8165f3623c48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:00.448950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71950a4e-91db-4789-b686-8165f3623c48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:00.462577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:00.475361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71950a4e-91db-4789-b686-8165f3623c48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:00.476962 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71950a4e-91db-4789-b686-8165f3623c48 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:00.483894 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16dd3d-25f4-456a-aebe-9f7535492abb None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:00.489884 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9847d36-b7bc-47f6-a4a9-c883c54b078e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28acb0322114aebafcf62bdf9a346df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:00.490044 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1130/2236] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:00.490795 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9847d36-b7bc-47f6-a4a9-c883c54b078e service swift] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:00.491215 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9847d36-b7bc-47f6-a4a9-c883c54b078e service swift] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:00.491498 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9847d36-b7bc-47f6-a4a9-c883c54b078e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:00.501355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9847d36-b7bc-47f6-a4a9-c883c54b078e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:00.503030 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9847d36-b7bc-47f6-a4a9-c883c54b078e service swift] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:00.513219 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1107/2237] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Jun 27 21:18:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:18:00.816437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4017f9-d6d2-48e2-b6af-3d8718175395 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:00.816952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4017f9-d6d2-48e2-b6af-3d8718175395 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:00.817053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4017f9-d6d2-48e2-b6af-3d8718175395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:00.837653 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-dc4017f9-d6d2-48e2-b6af-3d8718175395 None None] MFA Rules not processed for user `e802da0b078e457da466a9a7b0f15085`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:00.839307 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-dc4017f9-d6d2-48e2-b6af-3d8718175395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:00.846897 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1131/2238] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Jun 27 21:18:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:00.850794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9847d36-b7bc-47f6-a4a9-c883c54b078e service swift] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:00.852244 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e9847d36-b7bc-47f6-a4a9-c883c54b078e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:00.868489 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4017f9-d6d2-48e2-b6af-3d8718175395 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:00.871925 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9847d36-b7bc-47f6-a4a9-c883c54b078e service swift] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:00.879040 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5aa70-d0f8-47a1-ae6f-f5ab1c4f20c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:00.879215 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5aa70-d0f8-47a1-ae6f-f5ab1c4f20c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:00.879549 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5aa70-d0f8-47a1-ae6f-f5ab1c4f20c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:00.880193 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5aa70-d0f8-47a1-ae6f-f5ab1c4f20c8 admin admin] PATH_INFO: `/v3/projects/3410c5719ac1452a8ead4b5f6f0f536f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:00.881704 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5aa70-d0f8-47a1-ae6f-f5ab1c4f20c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3410c5719ac1452a8ead4b5f6f0f536f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:00.884056 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5aa70-d0f8-47a1-ae6f-f5ab1c4f20c8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:00.884810 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4017f9-d6d2-48e2-b6af-3d8718175395 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:00.893112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570cdc3-0f0f-455c-9a6c-ca6255d4941b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28acb0322114aebafcf62bdf9a346df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:00.901054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570cdc3-0f0f-455c-9a6c-ca6255d4941b service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:00.901534 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570cdc3-0f0f-455c-9a6c-ca6255d4941b service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:00.901985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570cdc3-0f0f-455c-9a6c-ca6255d4941b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:00.913570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570cdc3-0f0f-455c-9a6c-ca6255d4941b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:00.915645 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570cdc3-0f0f-455c-9a6c-ca6255d4941b service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:00.934575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1132/2239] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Jun 27 21:18:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:18:00.957462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570cdc3-0f0f-455c-9a6c-ca6255d4941b service swift] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:00.959751 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9570cdc3-0f0f-455c-9a6c-ca6255d4941b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:00.973238 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3de5aa70-d0f8-47a1-ae6f-f5ab1c4f20c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3410c5719ac1452a8ead4b5f6f0f536f', 'request_id': 'req-3de5aa70-d0f8-47a1-ae6f-f5ab1c4f20c8'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:00.985149 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3de5aa70-d0f8-47a1-ae6f-f5ab1c4f20c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3410c5719ac1452a8ead4b5f6f0f536f', 'request_id': 'req-3de5aa70-d0f8-47a1-ae6f-f5ab1c4f20c8'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:00.994426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3de5aa70-d0f8-47a1-ae6f-f5ab1c4f20c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3410c5719ac1452a8ead4b5f6f0f536f', 'request_id': 'req-3de5aa70-d0f8-47a1-ae6f-f5ab1c4f20c8'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:00.998868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570cdc3-0f0f-455c-9a6c-ca6255d4941b service swift] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:01.009365 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3de5aa70-d0f8-47a1-ae6f-f5ab1c4f20c8 admin admin] The token cache is being invalidate because project 3410c5719ac1452a8ead4b5f6f0f536f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:18:01.009923 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3de5aa70-d0f8-47a1-ae6f-f5ab1c4f20c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:01.010588 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3de5aa70-d0f8-47a1-ae6f-f5ab1c4f20c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:01.012223 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1108/2240] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:18:00 2022] DELETE /identity/v3/projects/3410c5719ac1452a8ead4b5f6f0f536f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:01.016534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5aa70-d0f8-47a1-ae6f-f5ab1c4f20c8 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:01.017918 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3de5aa70-d0f8-47a1-ae6f-f5ab1c4f20c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:01.027581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:01.081424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2670d0-3f59-45bd-8068-bf89e6029c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:01.082527 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2670d0-3f59-45bd-8068-bf89e6029c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:01.082811 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2670d0-3f59-45bd-8068-bf89e6029c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:01.083310 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2670d0-3f59-45bd-8068-bf89e6029c6f admin admin] PATH_INFO: `/v3/projects/78679f3700e04c8fa9ace9086eec127a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:01.084729 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2670d0-3f59-45bd-8068-bf89e6029c6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=78679f3700e04c8fa9ace9086eec127a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:01.086763 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2670d0-3f59-45bd-8068-bf89e6029c6f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:01.102789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5aa70-d0f8-47a1-ae6f-f5ab1c4f20c8 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:01.111243 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccfc86-c899-47db-8327-e4e8930c3c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:01.112257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccfc86-c899-47db-8327-e4e8930c3c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:01.112710 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccfc86-c899-47db-8327-e4e8930c3c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:01.113061 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccfc86-c899-47db-8327-e4e8930c3c78 admin admin] PATH_INFO: `/v3/users/6206f0ddf81b402fb4e3e325c984ec86` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:01.114877 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccfc86-c899-47db-8327-e4e8930c3c78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6206f0ddf81b402fb4e3e325c984ec86)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:01.116734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccfc86-c899-47db-8327-e4e8930c3c78 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:01.142526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ba2670d0-3f59-45bd-8068-bf89e6029c6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78679f3700e04c8fa9ace9086eec127a', 'request_id': 'req-ba2670d0-3f59-45bd-8068-bf89e6029c6f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:01.157570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ba2670d0-3f59-45bd-8068-bf89e6029c6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78679f3700e04c8fa9ace9086eec127a', 'request_id': 'req-ba2670d0-3f59-45bd-8068-bf89e6029c6f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:01.167668 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ba2670d0-3f59-45bd-8068-bf89e6029c6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78679f3700e04c8fa9ace9086eec127a', 'request_id': 'req-ba2670d0-3f59-45bd-8068-bf89e6029c6f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:01.186556 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-31ccfc86-c899-47db-8327-e4e8930c3c78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6206f0ddf81b402fb4e3e325c984ec86'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:01.187399 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ba2670d0-3f59-45bd-8068-bf89e6029c6f admin admin] The token cache is being invalidate because project 78679f3700e04c8fa9ace9086eec127a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:18:01.187940 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ba2670d0-3f59-45bd-8068-bf89e6029c6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:01.188160 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-31ccfc86-c899-47db-8327-e4e8930c3c78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6206f0ddf81b402fb4e3e325c984ec86'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:01.188993 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-31ccfc86-c899-47db-8327-e4e8930c3c78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6206f0ddf81b402fb4e3e325c984ec86'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:01.189503 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ba2670d0-3f59-45bd-8068-bf89e6029c6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:01.191798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1133/2241] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:18:00 2022] DELETE /identity/v3/projects/78679f3700e04c8fa9ace9086eec127a => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:18:01.197621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2dc650-790a-4265-a645-1a0a2488181d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:01.197903 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2dc650-790a-4265-a645-1a0a2488181d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:01.198205 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2dc650-790a-4265-a645-1a0a2488181d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:01.217280 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-31ccfc86-c899-47db-8327-e4e8930c3c78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6206f0ddf81b402fb4e3e325c984ec86'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:01.222921 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:01.226371 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6c2dc650-790a-4265-a645-1a0a2488181d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:01.226971 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-31ccfc86-c899-47db-8327-e4e8930c3c78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6206f0ddf81b402fb4e3e325c984ec86'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:01.227566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6c2dc650-790a-4265-a645-1a0a2488181d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:01.248667 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1109/2242] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:18:01 2022] DELETE /identity/v3/users/6206f0ddf81b402fb4e3e325c984ec86 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:18:01.317933 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1134/2243] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:18:01.331172 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468aae4-e5c2-4d47-9278-93ca9035d224 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:01.331480 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468aae4-e5c2-4d47-9278-93ca9035d224 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:01.332841 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468aae4-e5c2-4d47-9278-93ca9035d224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:01.352553 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e468aae4-e5c2-4d47-9278-93ca9035d224 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:01.354507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e468aae4-e5c2-4d47-9278-93ca9035d224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:01.389590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59e5f0-1458-4f9b-ba99-dbd15c3d5436 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:01.391399 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59e5f0-1458-4f9b-ba99-dbd15c3d5436 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:01.392002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59e5f0-1458-4f9b-ba99-dbd15c3d5436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:01.414239 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7a59e5f0-1458-4f9b-ba99-dbd15c3d5436 None None] MFA Rules not processed for user `e802da0b078e457da466a9a7b0f15085`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:01.415785 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7a59e5f0-1458-4f9b-ba99-dbd15c3d5436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:01.501741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:01.512005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1110/2244] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:01.516639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468aae4-e5c2-4d47-9278-93ca9035d224 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:01.520426 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1135/2245] 10.222.0.27 () {58 vars in 967 bytes} [Mon Jun 27 21:18:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:18:01.526878 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468aae4-e5c2-4d47-9278-93ca9035d224 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:01.532067 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028377b9-7e68-49d0-bdb7-1aaf406b9241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:01.532745 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028377b9-7e68-49d0-bdb7-1aaf406b9241 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:01.533035 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028377b9-7e68-49d0-bdb7-1aaf406b9241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:01.533318 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028377b9-7e68-49d0-bdb7-1aaf406b9241 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:01.534086 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028377b9-7e68-49d0-bdb7-1aaf406b9241 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:18:01.534397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028377b9-7e68-49d0-bdb7-1aaf406b9241 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:01.536159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028377b9-7e68-49d0-bdb7-1aaf406b9241 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:01.546498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59e5f0-1458-4f9b-ba99-dbd15c3d5436 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:01.547948 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7a59e5f0-1458-4f9b-ba99-dbd15c3d5436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:01.550183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1111/2246] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:18:01.557711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028377b9-7e68-49d0-bdb7-1aaf406b9241 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:01.579891 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028377b9-7e68-49d0-bdb7-1aaf406b9241 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:01.594326 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b0344-5e66-4623-b727-c324691a4a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:01.595295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b0344-5e66-4623-b727-c324691a4a88 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:01.595761 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b0344-5e66-4623-b727-c324691a4a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:01.596203 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b0344-5e66-4623-b727-c324691a4a88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:01.597056 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b0344-5e66-4623-b727-c324691a4a88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:01.599175 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b0344-5e66-4623-b727-c324691a4a88 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:01.650775 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59e5f0-1458-4f9b-ba99-dbd15c3d5436 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:01.654455 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1112/2247] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:18:01 2022] POST /identity/v3/projects => generated 397 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:18:01.661053 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b0344-5e66-4623-b727-c324691a4a88 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:01.790596 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3b6f6-62fe-4436-b969-4b801ae3c030 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28acb0322114aebafcf62bdf9a346df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:01.792417 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3b6f6-62fe-4436-b969-4b801ae3c030 service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:01.792417 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3b6f6-62fe-4436-b969-4b801ae3c030 service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:01.793142 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3b6f6-62fe-4436-b969-4b801ae3c030 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:01.826412 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b0344-5e66-4623-b727-c324691a4a88 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:01.843496 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c602fb7-5cff-4774-8216-0aa29da38c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:01.846177 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c602fb7-5cff-4774-8216-0aa29da38c0a admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:01.846783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c602fb7-5cff-4774-8216-0aa29da38c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:01.847207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c602fb7-5cff-4774-8216-0aa29da38c0a admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:01.848107 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c602fb7-5cff-4774-8216-0aa29da38c0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:01.850404 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c602fb7-5cff-4774-8216-0aa29da38c0a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:01.875663 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1113/2248] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:18:01 2022] POST /identity/v3/users => generated 299 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:18:01.879859 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c602fb7-5cff-4774-8216-0aa29da38c0a admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:01.885380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3b6f6-62fe-4436-b969-4b801ae3c030 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:01.887590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3b6f6-62fe-4436-b969-4b801ae3c030 service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:01.896277 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c602fb7-5cff-4774-8216-0aa29da38c0a admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:01.903942 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92e783-df99-4dd5-9e0b-d04fc1805bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:01.904707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92e783-df99-4dd5-9e0b-d04fc1805bbf admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:01.905048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92e783-df99-4dd5-9e0b-d04fc1805bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:01.905260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92e783-df99-4dd5-9e0b-d04fc1805bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:01.906404 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92e783-df99-4dd5-9e0b-d04fc1805bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:01.909098 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92e783-df99-4dd5-9e0b-d04fc1805bbf admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:01.915977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1136/2249] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Jun 27 21:18:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 21:18:01.923139 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1114/2250] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:18:01.927147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3b6f6-62fe-4436-b969-4b801ae3c030 service swift] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:01.940599 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3b6f6-62fe-4436-b969-4b801ae3c030 service swift] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:01.951456 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de6221-0dbe-4bbb-9ec4-4ebe2e530da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:01.955529 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de6221-0dbe-4bbb-9ec4-4ebe2e530da9 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:01.955959 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92e783-df99-4dd5-9e0b-d04fc1805bbf admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:01.956739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de6221-0dbe-4bbb-9ec4-4ebe2e530da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:01.956739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de6221-0dbe-4bbb-9ec4-4ebe2e530da9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:01.957596 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de6221-0dbe-4bbb-9ec4-4ebe2e530da9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:01.960249 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de6221-0dbe-4bbb-9ec4-4ebe2e530da9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:01.965857 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7c92e783-df99-4dd5-9e0b-d04fc1805bbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:02.002886 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1137/2251] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:18:02.006680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de6221-0dbe-4bbb-9ec4-4ebe2e530da9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:02.008181 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92e783-df99-4dd5-9e0b-d04fc1805bbf admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:02.020978 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a47449-3256-4428-861f-86ed25083b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:02.023340 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a47449-3256-4428-861f-86ed25083b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:02.023655 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a47449-3256-4428-861f-86ed25083b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:02.023766 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de6221-0dbe-4bbb-9ec4-4ebe2e530da9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:02.023930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a47449-3256-4428-861f-86ed25083b7d admin admin] PATH_INFO: `/v3/projects/9897ef22d42e4eba915df5a1f2a5ab13` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:02.029872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1833f-7608-47cc-92f6-da15f528e2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:02.030795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1833f-7608-47cc-92f6-da15f528e2b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:02.031083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1833f-7608-47cc-92f6-da15f528e2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:02.031191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a47449-3256-4428-861f-86ed25083b7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9897ef22d42e4eba915df5a1f2a5ab13)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:02.031786 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1833f-7608-47cc-92f6-da15f528e2b0 admin admin] PATH_INFO: `/v3/projects/2b7448aedbc44e1a9385f0b7817f3f4c/users/9530ed9b13744f709cf521c66962d798/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:02.033163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a47449-3256-4428-861f-86ed25083b7d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:02.051709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1833f-7608-47cc-92f6-da15f528e2b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b7448aedbc44e1a9385f0b7817f3f4c, user_id=9530ed9b13744f709cf521c66962d798, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:02.054438 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1833f-7608-47cc-92f6-da15f528e2b0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:02.075114 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1138/2252] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:18:02 2022] PUT /identity/v3/projects/2b7448aedbc44e1a9385f0b7817f3f4c/users/9530ed9b13744f709cf521c66962d798/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:02.086085 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32602d96-1d75-40fa-ac67-5d654179c769 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:02.086440 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32602d96-1d75-40fa-ac67-5d654179c769 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:02.086859 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32602d96-1d75-40fa-ac67-5d654179c769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:02.090405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:02.105742 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:02.138301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-24a47449-3256-4428-861f-86ed25083b7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9897ef22d42e4eba915df5a1f2a5ab13', 'request_id': 'req-24a47449-3256-4428-861f-86ed25083b7d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:02.140733 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-32602d96-1d75-40fa-ac67-5d654179c769 None None] MFA Rules not processed for user `9530ed9b13744f709cf521c66962d798`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:02.142387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-32602d96-1d75-40fa-ac67-5d654179c769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:02.157816 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-24a47449-3256-4428-861f-86ed25083b7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9897ef22d42e4eba915df5a1f2a5ab13', 'request_id': 'req-24a47449-3256-4428-861f-86ed25083b7d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:02.181196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-24a47449-3256-4428-861f-86ed25083b7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9897ef22d42e4eba915df5a1f2a5ab13', 'request_id': 'req-24a47449-3256-4428-861f-86ed25083b7d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:02.206433 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-24a47449-3256-4428-861f-86ed25083b7d admin admin] The token cache is being invalidate because project 9897ef22d42e4eba915df5a1f2a5ab13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:18:02.206748 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-24a47449-3256-4428-861f-86ed25083b7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:02.207464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-24a47449-3256-4428-861f-86ed25083b7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:02.209459 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1115/2253] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:18:01 2022] DELETE /identity/v3/projects/9897ef22d42e4eba915df5a1f2a5ab13 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:02.214089 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a47449-3256-4428-861f-86ed25083b7d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:02.216360 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-24a47449-3256-4428-861f-86ed25083b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:02.279844 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1139/2254] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:02.293901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32602d96-1d75-40fa-ac67-5d654179c769 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:02.296740 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-32602d96-1d75-40fa-ac67-5d654179c769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:02.331395 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a47449-3256-4428-861f-86ed25083b7d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:02.335480 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf86b3-e4a1-4161-b245-2c6877408c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:02.336508 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf86b3-e4a1-4161-b245-2c6877408c7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:02.336872 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf86b3-e4a1-4161-b245-2c6877408c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:02.337213 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf86b3-e4a1-4161-b245-2c6877408c7b admin admin] PATH_INFO: `/v3/users/53093b67f7e145539b18ef59e3e4e21b` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:02.339240 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf86b3-e4a1-4161-b245-2c6877408c7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=53093b67f7e145539b18ef59e3e4e21b)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:02.341982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf86b3-e4a1-4161-b245-2c6877408c7b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:02.397568 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32602d96-1d75-40fa-ac67-5d654179c769 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:02.404724 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3aaf86b3-e4a1-4161-b245-2c6877408c7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53093b67f7e145539b18ef59e3e4e21b'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:02.408726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3aaf86b3-e4a1-4161-b245-2c6877408c7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53093b67f7e145539b18ef59e3e4e21b'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:02.408726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3aaf86b3-e4a1-4161-b245-2c6877408c7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53093b67f7e145539b18ef59e3e4e21b'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:02.413196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0bf6b-c667-4106-b0d2-d9cbcc8be867 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:02.413196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0bf6b-c667-4106-b0d2-d9cbcc8be867 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:02.414227 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0bf6b-c667-4106-b0d2-d9cbcc8be867 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:02.414607 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0bf6b-c667-4106-b0d2-d9cbcc8be867 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:02.416657 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f2a0bf6b-c667-4106-b0d2-d9cbcc8be867 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:02.420559 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3aaf86b3-e4a1-4161-b245-2c6877408c7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53093b67f7e145539b18ef59e3e4e21b'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:02.429628 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3aaf86b3-e4a1-4161-b245-2c6877408c7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53093b67f7e145539b18ef59e3e4e21b'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:02.433792 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a0bf6b-c667-4106-b0d2-d9cbcc8be867 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:02.435864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a0bf6b-c667-4106-b0d2-d9cbcc8be867 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:02.450952 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1116/2255] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:18:02 2022] DELETE /identity/v3/users/53093b67f7e145539b18ef59e3e4e21b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:02.462115 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1140/2256] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:02.951348 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf86b3-e4a1-4161-b245-2c6877408c7b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:02.953724 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3aaf86b3-e4a1-4161-b245-2c6877408c7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:03.047669 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:03.114358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf86b3-e4a1-4161-b245-2c6877408c7b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:03.122597 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004410d4-7f49-419f-905c-b1b620a7a7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:03.125625 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004410d4-7f49-419f-905c-b1b620a7a7a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:03.126382 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004410d4-7f49-419f-905c-b1b620a7a7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:03.127781 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004410d4-7f49-419f-905c-b1b620a7a7a6 admin admin] PATH_INFO: `/v3/projects/ce41f01380424924bf480db36f7a4650` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:03.131003 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004410d4-7f49-419f-905c-b1b620a7a7a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce41f01380424924bf480db36f7a4650)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:03.136649 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004410d4-7f49-419f-905c-b1b620a7a7a6 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:03.232616 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-004410d4-7f49-419f-905c-b1b620a7a7a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce41f01380424924bf480db36f7a4650', 'request_id': 'req-004410d4-7f49-419f-905c-b1b620a7a7a6'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:03.243286 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-004410d4-7f49-419f-905c-b1b620a7a7a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce41f01380424924bf480db36f7a4650', 'request_id': 'req-004410d4-7f49-419f-905c-b1b620a7a7a6'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:03.254071 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-004410d4-7f49-419f-905c-b1b620a7a7a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce41f01380424924bf480db36f7a4650', 'request_id': 'req-004410d4-7f49-419f-905c-b1b620a7a7a6'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:03.278579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-004410d4-7f49-419f-905c-b1b620a7a7a6 admin admin] The token cache is being invalidate because project ce41f01380424924bf480db36f7a4650 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:18:03.279030 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-004410d4-7f49-419f-905c-b1b620a7a7a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:03.280602 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-004410d4-7f49-419f-905c-b1b620a7a7a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:03.284233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1117/2257] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:18:02 2022] DELETE /identity/v3/projects/ce41f01380424924bf480db36f7a4650 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:03.289952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a7c79-fdd5-4c92-a043-37f337d4ba56 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:03.290467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a7c79-fdd5-4c92-a043-37f337d4ba56 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:03.290729 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a7c79-fdd5-4c92-a043-37f337d4ba56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:03.322195 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-173a7c79-fdd5-4c92-a043-37f337d4ba56 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:03.324138 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-173a7c79-fdd5-4c92-a043-37f337d4ba56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:03.460594 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1141/2258] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:03.468665 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c0a9a-1f03-4edf-bd95-723b765b809f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:03.469022 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c0a9a-1f03-4edf-bd95-723b765b809f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:03.469466 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c0a9a-1f03-4edf-bd95-723b765b809f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:03.493200 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1b8c0a9a-1f03-4edf-bd95-723b765b809f None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:03.494980 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1b8c0a9a-1f03-4edf-bd95-723b765b809f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:03.505425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1118/2259] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:03.510189 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a7c79-fdd5-4c92-a043-37f337d4ba56 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:03.524638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a7c79-fdd5-4c92-a043-37f337d4ba56 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:03.531241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f825a7-e185-4c93-aa55-f886c85d3e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:03.531846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f825a7-e185-4c93-aa55-f886c85d3e1f admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:03.532127 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f825a7-e185-4c93-aa55-f886c85d3e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:03.532400 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f825a7-e185-4c93-aa55-f886c85d3e1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:03.533317 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f825a7-e185-4c93-aa55-f886c85d3e1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:18:03.533630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f825a7-e185-4c93-aa55-f886c85d3e1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:03.536508 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f825a7-e185-4c93-aa55-f886c85d3e1f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:03.569941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1142/2260] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:18:03.574478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c0a9a-1f03-4edf-bd95-723b765b809f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:03.584921 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c0a9a-1f03-4edf-bd95-723b765b809f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:03.590117 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18f962-db7b-4bbb-8df5-7baf49893b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:03.590856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18f962-db7b-4bbb-8df5-7baf49893b54 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:03.591353 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18f962-db7b-4bbb-8df5-7baf49893b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:03.591728 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18f962-db7b-4bbb-8df5-7baf49893b54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:03.592520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18f962-db7b-4bbb-8df5-7baf49893b54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:03.594605 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18f962-db7b-4bbb-8df5-7baf49893b54 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:03.627254 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1119/2261] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:18:03 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:18:03.630911 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f825a7-e185-4c93-aa55-f886c85d3e1f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:03.642964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f825a7-e185-4c93-aa55-f886c85d3e1f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:03.683897 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:03.721454 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c36bf2-176c-499b-9e3e-075b60817a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:03.722228 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c36bf2-176c-499b-9e3e-075b60817a88 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:03.722576 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c36bf2-176c-499b-9e3e-075b60817a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:03.722919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c36bf2-176c-499b-9e3e-075b60817a88 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:03.723789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c36bf2-176c-499b-9e3e-075b60817a88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:03.725899 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c36bf2-176c-499b-9e3e-075b60817a88 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:03.742986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbb484-86fd-4f97-a764-8a717c5ec1e3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:03.743316 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbb484-86fd-4f97-a764-8a717c5ec1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:03.743597 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbb484-86fd-4f97-a764-8a717c5ec1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:03.749501 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1143/2262] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:18:03 2022] POST /identity/v3/users => generated 285 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:18:03.754699 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c36bf2-176c-499b-9e3e-075b60817a88 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:03.762046 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d0bbb484-86fd-4f97-a764-8a717c5ec1e3 None None] MFA Rules not processed for user `30cbde26a1964f22902c24917006fb90`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:03.763269 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d0bbb484-86fd-4f97-a764-8a717c5ec1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:03.765079 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c36bf2-176c-499b-9e3e-075b60817a88 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:03.770326 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059cf3d-eb07-48a5-9962-1fc5351a3a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:03.771074 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059cf3d-eb07-48a5-9962-1fc5351a3a11 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:03.771352 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059cf3d-eb07-48a5-9962-1fc5351a3a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:03.771643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059cf3d-eb07-48a5-9962-1fc5351a3a11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:03.772374 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059cf3d-eb07-48a5-9962-1fc5351a3a11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:03.774010 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059cf3d-eb07-48a5-9962-1fc5351a3a11 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:03.785835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1144/2263] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:18:03.789796 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059cf3d-eb07-48a5-9962-1fc5351a3a11 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:03.800396 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059cf3d-eb07-48a5-9962-1fc5351a3a11 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:03.806349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2b462-3d22-4039-8f0d-9e65bf7f5762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:03.807127 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2b462-3d22-4039-8f0d-9e65bf7f5762 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:03.807479 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2b462-3d22-4039-8f0d-9e65bf7f5762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:03.807854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2b462-3d22-4039-8f0d-9e65bf7f5762 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:03.808691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2b462-3d22-4039-8f0d-9e65bf7f5762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:03.810527 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2b462-3d22-4039-8f0d-9e65bf7f5762 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:03.821772 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1120/2264] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:18:03.821943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1145/2265] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:18:03.825834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bbb484-86fd-4f97-a764-8a717c5ec1e3 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:03.836703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bbb484-86fd-4f97-a764-8a717c5ec1e3 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:03.841431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbab47-9649-4965-97a7-b68ec2d3c2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:03.842329 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbab47-9649-4965-97a7-b68ec2d3c2cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:03.842614 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbab47-9649-4965-97a7-b68ec2d3c2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:03.842893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbab47-9649-4965-97a7-b68ec2d3c2cb admin admin] PATH_INFO: `/v3/projects/0f10581267df454c93cf3554fb633e3d/users/0c2e0d2445714a568580da468ce7d087/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:03.857998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbab47-9649-4965-97a7-b68ec2d3c2cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f10581267df454c93cf3554fb633e3d, user_id=0c2e0d2445714a568580da468ce7d087, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:03.860727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbab47-9649-4965-97a7-b68ec2d3c2cb admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:03.876386 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1121/2266] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:18:03 2022] PUT /identity/v3/projects/0f10581267df454c93cf3554fb633e3d/users/0c2e0d2445714a568580da468ce7d087/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:18:03.887224 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438e92e-07a9-4d9a-a40d-58b7400d68c6 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:03.887519 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438e92e-07a9-4d9a-a40d-58b7400d68c6 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:03.887859 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438e92e-07a9-4d9a-a40d-58b7400d68c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:03.915284 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c438e92e-07a9-4d9a-a40d-58b7400d68c6 None None] MFA Rules not processed for user `0c2e0d2445714a568580da468ce7d087`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:03.916774 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c438e92e-07a9-4d9a-a40d-58b7400d68c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:03.986715 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1146/2267] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:03 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:03.998968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbab47-9649-4965-97a7-b68ec2d3c2cb admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:04.000804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-57bbab47-9649-4965-97a7-b68ec2d3c2cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:04.019492 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:04.063151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbab47-9649-4965-97a7-b68ec2d3c2cb admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:04.067017 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871f483-3987-42f1-bb34-61b0f12eaa52 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b43db1fd0b494a3f8588b498b585f247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:04.067765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871f483-3987-42f1-bb34-61b0f12eaa52 service glance] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:04.068054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871f483-3987-42f1-bb34-61b0f12eaa52 service glance] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:04.068331 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871f483-3987-42f1-bb34-61b0f12eaa52 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:04.077288 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c871f483-3987-42f1-bb34-61b0f12eaa52 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:04.078730 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c871f483-3987-42f1-bb34-61b0f12eaa52 service glance] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:04.088011 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1122/2268] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:03 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:04.740387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438e92e-07a9-4d9a-a40d-58b7400d68c6 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:04.743407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c438e92e-07a9-4d9a-a40d-58b7400d68c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:04.834240 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438e92e-07a9-4d9a-a40d-58b7400d68c6 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:04.839203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90964fb2-0387-4889-8300-608de0049ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:04.840137 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90964fb2-0387-4889-8300-608de0049ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:04.840548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90964fb2-0387-4889-8300-608de0049ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:04.841156 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90964fb2-0387-4889-8300-608de0049ee3 admin admin] PATH_INFO: `/v3/users/cc12ebd7d75b460f9e520efcbad1aa11` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:04.842029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90964fb2-0387-4889-8300-608de0049ee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc12ebd7d75b460f9e520efcbad1aa11)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:04.844066 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90964fb2-0387-4889-8300-608de0049ee3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:04.879349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:04.900961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-90964fb2-0387-4889-8300-608de0049ee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc12ebd7d75b460f9e520efcbad1aa11'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:04.901798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-90964fb2-0387-4889-8300-608de0049ee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc12ebd7d75b460f9e520efcbad1aa11'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:04.902482 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-90964fb2-0387-4889-8300-608de0049ee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc12ebd7d75b460f9e520efcbad1aa11'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:04.914641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-90964fb2-0387-4889-8300-608de0049ee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc12ebd7d75b460f9e520efcbad1aa11'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:04.922161 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-90964fb2-0387-4889-8300-608de0049ee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc12ebd7d75b460f9e520efcbad1aa11'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:04.937325 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1147/2269] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:18:04 2022] DELETE /identity/v3/users/cc12ebd7d75b460f9e520efcbad1aa11 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:04.942021 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59514ed-e99d-4859-9cb4-732ff52c5316 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:04.942413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59514ed-e99d-4859-9cb4-732ff52c5316 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:04.942810 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59514ed-e99d-4859-9cb4-732ff52c5316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:04.961273 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e59514ed-e99d-4859-9cb4-732ff52c5316 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:04.962487 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e59514ed-e99d-4859-9cb4-732ff52c5316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:05.025506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1123/2270] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:05.036023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90964fb2-0387-4889-8300-608de0049ee3 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:05.038566 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-90964fb2-0387-4889-8300-608de0049ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:05.106647 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90964fb2-0387-4889-8300-608de0049ee3 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:05.110721 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c378ad1-0055-400b-8a55-14e46ad05f53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:05.111348 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c378ad1-0055-400b-8a55-14e46ad05f53 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:05.111648 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c378ad1-0055-400b-8a55-14e46ad05f53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:05.111926 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c378ad1-0055-400b-8a55-14e46ad05f53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:05.122177 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c378ad1-0055-400b-8a55-14e46ad05f53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:05.123692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c378ad1-0055-400b-8a55-14e46ad05f53 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:05.133469 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1148/2271] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:05.310377 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59514ed-e99d-4859-9cb4-732ff52c5316 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:05.312194 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e59514ed-e99d-4859-9cb4-732ff52c5316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:05.332242 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59514ed-e99d-4859-9cb4-732ff52c5316 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:05.345966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf48af-6616-4309-ac77-20481b344427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:05.347057 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf48af-6616-4309-ac77-20481b344427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:05.347495 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf48af-6616-4309-ac77-20481b344427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:05.347874 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf48af-6616-4309-ac77-20481b344427 admin admin] PATH_INFO: `/v3/projects/27b2158e24b0466c9b344c4f8c4f1301` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:05.349075 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf48af-6616-4309-ac77-20481b344427 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27b2158e24b0466c9b344c4f8c4f1301)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:05.351291 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf48af-6616-4309-ac77-20481b344427 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:05.410662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-acbf48af-6616-4309-ac77-20481b344427 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27b2158e24b0466c9b344c4f8c4f1301', 'request_id': 'req-acbf48af-6616-4309-ac77-20481b344427'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:05.419538 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-acbf48af-6616-4309-ac77-20481b344427 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27b2158e24b0466c9b344c4f8c4f1301', 'request_id': 'req-acbf48af-6616-4309-ac77-20481b344427'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:05.425338 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-acbf48af-6616-4309-ac77-20481b344427 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27b2158e24b0466c9b344c4f8c4f1301', 'request_id': 'req-acbf48af-6616-4309-ac77-20481b344427'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:05.439739 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-acbf48af-6616-4309-ac77-20481b344427 admin admin] The token cache is being invalidate because project 27b2158e24b0466c9b344c4f8c4f1301 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:18:05.440172 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-acbf48af-6616-4309-ac77-20481b344427 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:05.441049 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-acbf48af-6616-4309-ac77-20481b344427 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:05.443228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1124/2272] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:18:05 2022] DELETE /identity/v3/projects/27b2158e24b0466c9b344c4f8c4f1301 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:10.306499 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c378ad1-0055-400b-8a55-14e46ad05f53 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:10.308898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4c378ad1-0055-400b-8a55-14e46ad05f53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:10.324021 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c378ad1-0055-400b-8a55-14e46ad05f53 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:10.404365 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be45ff-318b-44b5-976d-6ed4502d3722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:10.406150 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be45ff-318b-44b5-976d-6ed4502d3722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:10.406374 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be45ff-318b-44b5-976d-6ed4502d3722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:10.406791 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be45ff-318b-44b5-976d-6ed4502d3722 admin admin] PATH_INFO: `/v3/users/91fcd1d4ce2c49e6a67121e9c5398a35` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:10.408633 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be45ff-318b-44b5-976d-6ed4502d3722 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91fcd1d4ce2c49e6a67121e9c5398a35)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:10.411361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be45ff-318b-44b5-976d-6ed4502d3722 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:10.483544 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-84be45ff-318b-44b5-976d-6ed4502d3722 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91fcd1d4ce2c49e6a67121e9c5398a35'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:10.484296 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-84be45ff-318b-44b5-976d-6ed4502d3722 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91fcd1d4ce2c49e6a67121e9c5398a35'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:10.484908 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-84be45ff-318b-44b5-976d-6ed4502d3722 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91fcd1d4ce2c49e6a67121e9c5398a35'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:10.486353 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:10.504951 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-84be45ff-318b-44b5-976d-6ed4502d3722 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91fcd1d4ce2c49e6a67121e9c5398a35'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:10.513107 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-84be45ff-318b-44b5-976d-6ed4502d3722 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91fcd1d4ce2c49e6a67121e9c5398a35'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:10.527667 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1149/2273] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:18:10 2022] DELETE /identity/v3/users/91fcd1d4ce2c49e6a67121e9c5398a35 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:18:10.532931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924837d-3017-41b0-be34-61134b3d0773 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:10.533438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924837d-3017-41b0-be34-61134b3d0773 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:10.533576 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924837d-3017-41b0-be34-61134b3d0773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:10.554417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-2924837d-3017-41b0-be34-61134b3d0773 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:10.556313 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2924837d-3017-41b0-be34-61134b3d0773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:10.596643 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:10.619464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1125/2274] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:10.629102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be45ff-318b-44b5-976d-6ed4502d3722 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:10.631656 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-84be45ff-318b-44b5-976d-6ed4502d3722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:10.695240 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be45ff-318b-44b5-976d-6ed4502d3722 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:10.698981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a43358-6574-4085-bd58-e7fa2336cd4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:10.699696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a43358-6574-4085-bd58-e7fa2336cd4d service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:10.699978 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a43358-6574-4085-bd58-e7fa2336cd4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:10.700250 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a43358-6574-4085-bd58-e7fa2336cd4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:10.709606 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a43358-6574-4085-bd58-e7fa2336cd4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:10.711045 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a43358-6574-4085-bd58-e7fa2336cd4d service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:10.720850 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1150/2275] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:11.121853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924837d-3017-41b0-be34-61134b3d0773 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:11.123545 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2924837d-3017-41b0-be34-61134b3d0773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:11.136477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924837d-3017-41b0-be34-61134b3d0773 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:11.143409 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3d4aa-5fd2-4720-a34a-0eaeb1b9b8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:11.144511 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3d4aa-5fd2-4720-a34a-0eaeb1b9b8fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:11.144789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3d4aa-5fd2-4720-a34a-0eaeb1b9b8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:11.145066 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3d4aa-5fd2-4720-a34a-0eaeb1b9b8fb admin admin] PATH_INFO: `/v3/users/9530ed9b13744f709cf521c66962d798` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:11.146408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3d4aa-5fd2-4720-a34a-0eaeb1b9b8fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9530ed9b13744f709cf521c66962d798)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:11.148596 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3d4aa-5fd2-4720-a34a-0eaeb1b9b8fb admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:11.180691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a43358-6574-4085-bd58-e7fa2336cd4d service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:11.182335 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-71a43358-6574-4085-bd58-e7fa2336cd4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:11.197135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a43358-6574-4085-bd58-e7fa2336cd4d service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:11.207750 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9a087-c59f-4cf1-97ec-ac123e795a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:11.209509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9a087-c59f-4cf1-97ec-ac123e795a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:11.209509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9a087-c59f-4cf1-97ec-ac123e795a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:11.209741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9a087-c59f-4cf1-97ec-ac123e795a37 admin admin] PATH_INFO: `/v3/projects/cf54b3443056486f9575a111d25d928e` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:11.214823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9a087-c59f-4cf1-97ec-ac123e795a37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf54b3443056486f9575a111d25d928e)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:11.217201 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9a087-c59f-4cf1-97ec-ac123e795a37 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:11.237810 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-17a3d4aa-5fd2-4720-a34a-0eaeb1b9b8fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9530ed9b13744f709cf521c66962d798'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:11.238461 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-17a3d4aa-5fd2-4720-a34a-0eaeb1b9b8fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9530ed9b13744f709cf521c66962d798'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:11.239060 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-17a3d4aa-5fd2-4720-a34a-0eaeb1b9b8fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9530ed9b13744f709cf521c66962d798'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:11.251756 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-17a3d4aa-5fd2-4720-a34a-0eaeb1b9b8fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9530ed9b13744f709cf521c66962d798'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:11.259103 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-17a3d4aa-5fd2-4720-a34a-0eaeb1b9b8fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9530ed9b13744f709cf521c66962d798'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:11.269324 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-eed9a087-c59f-4cf1-97ec-ac123e795a37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf54b3443056486f9575a111d25d928e', 'request_id': 'req-eed9a087-c59f-4cf1-97ec-ac123e795a37'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:11.274315 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1126/2276] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:18:11 2022] DELETE /identity/v3/users/9530ed9b13744f709cf521c66962d798 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:11.277333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-eed9a087-c59f-4cf1-97ec-ac123e795a37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf54b3443056486f9575a111d25d928e', 'request_id': 'req-eed9a087-c59f-4cf1-97ec-ac123e795a37'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:11.278807 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd04fa-ee5c-4049-8dc5-1e1e7210f470 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:11.279099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd04fa-ee5c-4049-8dc5-1e1e7210f470 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:11.279378 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd04fa-ee5c-4049-8dc5-1e1e7210f470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:11.283235 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-eed9a087-c59f-4cf1-97ec-ac123e795a37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf54b3443056486f9575a111d25d928e', 'request_id': 'req-eed9a087-c59f-4cf1-97ec-ac123e795a37'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:11.294938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-eed9a087-c59f-4cf1-97ec-ac123e795a37 admin admin] The token cache is being invalidate because project cf54b3443056486f9575a111d25d928e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:18:11.295157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-eed9a087-c59f-4cf1-97ec-ac123e795a37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:11.295823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-eed9a087-c59f-4cf1-97ec-ac123e795a37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:11.296157 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-71dd04fa-ee5c-4049-8dc5-1e1e7210f470 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:11.297631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-71dd04fa-ee5c-4049-8dc5-1e1e7210f470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:11.297748 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1151/2277] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:18:11 2022] DELETE /identity/v3/projects/cf54b3443056486f9575a111d25d928e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:11.302383 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c747d-c3b8-4dea-a8a5-36ea497a4375 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:11.302671 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c747d-c3b8-4dea-a8a5-36ea497a4375 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:11.303102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c747d-c3b8-4dea-a8a5-36ea497a4375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:11.322732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ef9c747d-c3b8-4dea-a8a5-36ea497a4375 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:11.324002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ef9c747d-c3b8-4dea-a8a5-36ea497a4375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:11.371643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:11.373744 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1127/2278] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:18:11.383218 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd04fa-ee5c-4049-8dc5-1e1e7210f470 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:11.384797 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-71dd04fa-ee5c-4049-8dc5-1e1e7210f470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:11.388413 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1152/2279] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:18:11.396680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddee58a-710f-4ab8-88dd-413c30c8fbe3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:11.396970 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddee58a-710f-4ab8-88dd-413c30c8fbe3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:11.397229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddee58a-710f-4ab8-88dd-413c30c8fbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:11.417725 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-eddee58a-710f-4ab8-88dd-413c30c8fbe3 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:11.419652 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-eddee58a-710f-4ab8-88dd-413c30c8fbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:11.435156 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1153/2280] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:11.439183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddee58a-710f-4ab8-88dd-413c30c8fbe3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:11.449226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd04fa-ee5c-4049-8dc5-1e1e7210f470 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:11.449956 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddee58a-710f-4ab8-88dd-413c30c8fbe3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:11.452900 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba5a2a-dc3b-4a3c-9666-0394231c8805 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:11.453557 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba5a2a-dc3b-4a3c-9666-0394231c8805 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:11.453840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba5a2a-dc3b-4a3c-9666-0394231c8805 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:11.454113 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba5a2a-dc3b-4a3c-9666-0394231c8805 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:11.456678 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef950a6-e496-403b-b22b-f02f358eb073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:11.457807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef950a6-e496-403b-b22b-f02f358eb073 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:11.458318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef950a6-e496-403b-b22b-f02f358eb073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:11.458746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef950a6-e496-403b-b22b-f02f358eb073 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:11.459492 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef950a6-e496-403b-b22b-f02f358eb073 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:18:11.459853 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef950a6-e496-403b-b22b-f02f358eb073 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:11.461839 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef950a6-e496-403b-b22b-f02f358eb073 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:11.469911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba5a2a-dc3b-4a3c-9666-0394231c8805 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:11.472490 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba5a2a-dc3b-4a3c-9666-0394231c8805 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:11.476992 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:11.483184 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1154/2281] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:18:11.484878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef950a6-e496-403b-b22b-f02f358eb073 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:11.492720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1128/2282] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:18:11.502071 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef950a6-e496-403b-b22b-f02f358eb073 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:11.506878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c72df1-3341-4f60-8654-986d36a4b989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:11.507551 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c72df1-3341-4f60-8654-986d36a4b989 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:11.507862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c72df1-3341-4f60-8654-986d36a4b989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:11.508135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c72df1-3341-4f60-8654-986d36a4b989 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:11.509227 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c72df1-3341-4f60-8654-986d36a4b989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:11.511470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c72df1-3341-4f60-8654-986d36a4b989 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:11.553155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1155/2283] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:18:11 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:18:11.554337 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba5a2a-dc3b-4a3c-9666-0394231c8805 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:11.569691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba5a2a-dc3b-4a3c-9666-0394231c8805 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:11.651233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff5427-669b-4cf8-8bd2-1593a3224e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:11.652416 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff5427-669b-4cf8-8bd2-1593a3224e44 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:11.652416 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff5427-669b-4cf8-8bd2-1593a3224e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:11.652616 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff5427-669b-4cf8-8bd2-1593a3224e44 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:11.653250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff5427-669b-4cf8-8bd2-1593a3224e44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:11.655367 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff5427-669b-4cf8-8bd2-1593a3224e44 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:11.686493 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1129/2284] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:18:11 2022] POST /identity/v3/users => generated 286 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:18:11.690241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c72df1-3341-4f60-8654-986d36a4b989 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:11.701202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c72df1-3341-4f60-8654-986d36a4b989 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:11.705838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe7bbc-06ae-4f2c-b70c-33f6e807dbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:11.706501 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe7bbc-06ae-4f2c-b70c-33f6e807dbdc admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:11.706787 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe7bbc-06ae-4f2c-b70c-33f6e807dbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:11.707058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe7bbc-06ae-4f2c-b70c-33f6e807dbdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:11.707822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe7bbc-06ae-4f2c-b70c-33f6e807dbdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:11.709423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe7bbc-06ae-4f2c-b70c-33f6e807dbdc admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:11.749947 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1156/2285] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:18:11.750558 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff5427-669b-4cf8-8bd2-1593a3224e44 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:11.760419 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff5427-669b-4cf8-8bd2-1593a3224e44 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:11.765448 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2f076-f8ea-4f25-8453-8dbaee316239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:11.766123 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2f076-f8ea-4f25-8453-8dbaee316239 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:11.766408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2f076-f8ea-4f25-8453-8dbaee316239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:11.766683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2f076-f8ea-4f25-8453-8dbaee316239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:11.767415 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2f076-f8ea-4f25-8453-8dbaee316239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:11.769077 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2f076-f8ea-4f25-8453-8dbaee316239 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:11.779975 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1130/2286] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:18:11.783928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe7bbc-06ae-4f2c-b70c-33f6e807dbdc admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:11.794729 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe7bbc-06ae-4f2c-b70c-33f6e807dbdc admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:11.800008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe8da2-d664-4117-89ef-e50b117791d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:11.801012 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe8da2-d664-4117-89ef-e50b117791d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:11.801337 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe8da2-d664-4117-89ef-e50b117791d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:11.801752 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe8da2-d664-4117-89ef-e50b117791d1 admin admin] PATH_INFO: `/v3/projects/956b474bf9d249e5ba458f5f04da84eb/users/050c3a05ec004ae79e7d7ba2ffb4a53e/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:11.818720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe8da2-d664-4117-89ef-e50b117791d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=956b474bf9d249e5ba458f5f04da84eb, user_id=050c3a05ec004ae79e7d7ba2ffb4a53e, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:11.821627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe8da2-d664-4117-89ef-e50b117791d1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:11.846745 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1157/2287] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:18:11 2022] PUT /identity/v3/projects/956b474bf9d249e5ba458f5f04da84eb/users/050c3a05ec004ae79e7d7ba2ffb4a53e/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:11.857233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0acaef-404f-4b3f-9491-f2179b264a3e None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:11.857522 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0acaef-404f-4b3f-9491-f2179b264a3e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:11.857824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0acaef-404f-4b3f-9491-f2179b264a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:11.874397 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe8da2-d664-4117-89ef-e50b117791d1 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:11.876352 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2dbe8da2-d664-4117-89ef-e50b117791d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:11.886737 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-fd0acaef-404f-4b3f-9491-f2179b264a3e None None] MFA Rules not processed for user `050c3a05ec004ae79e7d7ba2ffb4a53e`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:11.887761 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fd0acaef-404f-4b3f-9491-f2179b264a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:11.953881 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe8da2-d664-4117-89ef-e50b117791d1 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:11.958544 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b888f-2713-4433-99b3-ff4176374980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:11.959416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b888f-2713-4433-99b3-ff4176374980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:11.959744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b888f-2713-4433-99b3-ff4176374980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:11.960016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b888f-2713-4433-99b3-ff4176374980 admin admin] PATH_INFO: `/v3/projects/2b7448aedbc44e1a9385f0b7817f3f4c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:11.961232 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b888f-2713-4433-99b3-ff4176374980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b7448aedbc44e1a9385f0b7817f3f4c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:11.963190 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b888f-2713-4433-99b3-ff4176374980 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:11.970987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1131/2288] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:11.993444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0acaef-404f-4b3f-9491-f2179b264a3e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:11.993444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fd0acaef-404f-4b3f-9491-f2179b264a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:12.050026 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0a4b888f-2713-4433-99b3-ff4176374980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b7448aedbc44e1a9385f0b7817f3f4c', 'request_id': 'req-0a4b888f-2713-4433-99b3-ff4176374980'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:12.060978 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0a4b888f-2713-4433-99b3-ff4176374980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b7448aedbc44e1a9385f0b7817f3f4c', 'request_id': 'req-0a4b888f-2713-4433-99b3-ff4176374980'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:12.069131 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0a4b888f-2713-4433-99b3-ff4176374980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b7448aedbc44e1a9385f0b7817f3f4c', 'request_id': 'req-0a4b888f-2713-4433-99b3-ff4176374980'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:12.069299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0acaef-404f-4b3f-9491-f2179b264a3e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:12.081590 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0a4b888f-2713-4433-99b3-ff4176374980 admin admin] The token cache is being invalidate because project 2b7448aedbc44e1a9385f0b7817f3f4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:18:12.082157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0a4b888f-2713-4433-99b3-ff4176374980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:12.083296 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0a4b888f-2713-4433-99b3-ff4176374980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:12.084591 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1158/2289] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:18:11 2022] DELETE /identity/v3/projects/2b7448aedbc44e1a9385f0b7817f3f4c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:18:12.089517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c383a-9a9c-4e79-906f-81625033f0fc None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:12.089872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c383a-9a9c-4e79-906f-81625033f0fc None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:12.090223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c383a-9a9c-4e79-906f-81625033f0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:12.097899 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:12.099256 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:12.113648 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7c5c383a-9a9c-4e79-906f-81625033f0fc None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:12.115173 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7c5c383a-9a9c-4e79-906f-81625033f0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:12.146706 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9b3c4-5564-4daa-9057-b552a65656d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:12.147367 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9b3c4-5564-4daa-9057-b552a65656d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:12.147680 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9b3c4-5564-4daa-9057-b552a65656d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:12.147956 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9b3c4-5564-4daa-9057-b552a65656d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:12.149806 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-47b9b3c4-5564-4daa-9057-b552a65656d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:12.188583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1159/2290] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:18:12.199558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823e742-9654-4ef9-ad24-2c98a56fa26b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:12.199911 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823e742-9654-4ef9-ad24-2c98a56fa26b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:12.200620 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823e742-9654-4ef9-ad24-2c98a56fa26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:12.224354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3823e742-9654-4ef9-ad24-2c98a56fa26b None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:12.225592 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3823e742-9654-4ef9-ad24-2c98a56fa26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:12.230906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b9b3c4-5564-4daa-9057-b552a65656d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:12.233198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b9b3c4-5564-4daa-9057-b552a65656d7 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:12.234487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1160/2291] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:12.238652 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823e742-9654-4ef9-ad24-2c98a56fa26b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:12.243954 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1132/2292] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:18:12.248265 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823e742-9654-4ef9-ad24-2c98a56fa26b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:12.252991 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180a971-6367-4996-ace9-488ecb7fd5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:12.253655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180a971-6367-4996-ace9-488ecb7fd5f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:12.254057 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180a971-6367-4996-ace9-488ecb7fd5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:12.254437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180a971-6367-4996-ace9-488ecb7fd5f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:12.255199 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180a971-6367-4996-ace9-488ecb7fd5f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:18:12.255514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180a971-6367-4996-ace9-488ecb7fd5f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:12.257179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180a971-6367-4996-ace9-488ecb7fd5f1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:12.268441 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9b3c4-5564-4daa-9057-b552a65656d7 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:12.269871 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-47b9b3c4-5564-4daa-9057-b552a65656d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:12.271519 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1161/2293] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:18:12.275709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180a971-6367-4996-ace9-488ecb7fd5f1 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:12.283439 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9b3c4-5564-4daa-9057-b552a65656d7 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:12.285870 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180a971-6367-4996-ace9-488ecb7fd5f1 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:12.288085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f0840-e56c-4567-b1e5-fa6675bece53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:12.288955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f0840-e56c-4567-b1e5-fa6675bece53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:12.289240 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f0840-e56c-4567-b1e5-fa6675bece53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:12.289514 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f0840-e56c-4567-b1e5-fa6675bece53 admin admin] PATH_INFO: `/v3/users/050c3a05ec004ae79e7d7ba2ffb4a53e` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:12.290716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f0840-e56c-4567-b1e5-fa6675bece53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=050c3a05ec004ae79e7d7ba2ffb4a53e)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:12.290896 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179ac1b-0369-43ed-9dce-e1a81e309f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:12.291543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179ac1b-0369-43ed-9dce-e1a81e309f40 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:12.291858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179ac1b-0369-43ed-9dce-e1a81e309f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:12.292138 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179ac1b-0369-43ed-9dce-e1a81e309f40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:12.292702 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f0840-e56c-4567-b1e5-fa6675bece53 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:12.292827 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179ac1b-0369-43ed-9dce-e1a81e309f40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:12.294735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179ac1b-0369-43ed-9dce-e1a81e309f40 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:12.326973 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1162/2294] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:18:12 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:18:12.327769 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179ac1b-0369-43ed-9dce-e1a81e309f40 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:12.352208 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-401f0840-e56c-4567-b1e5-fa6675bece53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '050c3a05ec004ae79e7d7ba2ffb4a53e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:12.352990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-401f0840-e56c-4567-b1e5-fa6675bece53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '050c3a05ec004ae79e7d7ba2ffb4a53e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:12.353666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-401f0840-e56c-4567-b1e5-fa6675bece53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '050c3a05ec004ae79e7d7ba2ffb4a53e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:12.366951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-401f0840-e56c-4567-b1e5-fa6675bece53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '050c3a05ec004ae79e7d7ba2ffb4a53e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:12.374094 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-401f0840-e56c-4567-b1e5-fa6675bece53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '050c3a05ec004ae79e7d7ba2ffb4a53e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:12.387655 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1133/2295] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:18:12 2022] DELETE /identity/v3/users/050c3a05ec004ae79e7d7ba2ffb4a53e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:12.391531 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179ac1b-0369-43ed-9dce-e1a81e309f40 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:12.392810 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccc3f0-92c1-4be8-ac10-6934928e53d0 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:12.393316 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccc3f0-92c1-4be8-ac10-6934928e53d0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:12.393600 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccc3f0-92c1-4be8-ac10-6934928e53d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:12.412221 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-84ccc3f0-92c1-4be8-ac10-6934928e53d0 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:12.413415 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-84ccc3f0-92c1-4be8-ac10-6934928e53d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:12.455226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b185756-3a82-4a50-a1ff-4a125899d3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:12.456419 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b185756-3a82-4a50-a1ff-4a125899d3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:12.456720 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b185756-3a82-4a50-a1ff-4a125899d3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:12.457005 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b185756-3a82-4a50-a1ff-4a125899d3e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:12.457680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b185756-3a82-4a50-a1ff-4a125899d3e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:12.459666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b185756-3a82-4a50-a1ff-4a125899d3e6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:12.465702 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:12.475918 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:12.481701 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1163/2296] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:18:12 2022] POST /identity/v3/users => generated 290 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:18:12.485588 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b185756-3a82-4a50-a1ff-4a125899d3e6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:12.489800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1134/2297] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:12.489965 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3b185756-3a82-4a50-a1ff-4a125899d3e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:12.496203 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccc3f0-92c1-4be8-ac10-6934928e53d0 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:12.497689 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-84ccc3f0-92c1-4be8-ac10-6934928e53d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:12.502574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b185756-3a82-4a50-a1ff-4a125899d3e6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:12.507781 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06804ed3-fe5f-495f-b73e-acaeafa29b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:12.508459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06804ed3-fe5f-495f-b73e-acaeafa29b61 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:12.508743 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06804ed3-fe5f-495f-b73e-acaeafa29b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:12.509016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06804ed3-fe5f-495f-b73e-acaeafa29b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:12.509752 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06804ed3-fe5f-495f-b73e-acaeafa29b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:12.511368 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06804ed3-fe5f-495f-b73e-acaeafa29b61 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:12.525851 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1164/2298] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:18:12.527134 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06804ed3-fe5f-495f-b73e-acaeafa29b61 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:12.546290 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06804ed3-fe5f-495f-b73e-acaeafa29b61 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:12.551565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d877b-67e8-4ef5-869d-00d7021f9884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:12.552260 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d877b-67e8-4ef5-869d-00d7021f9884 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:12.552613 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d877b-67e8-4ef5-869d-00d7021f9884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:12.552881 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d877b-67e8-4ef5-869d-00d7021f9884 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:12.553633 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d877b-67e8-4ef5-869d-00d7021f9884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:12.555343 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d877b-67e8-4ef5-869d-00d7021f9884 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:12.566185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccc3f0-92c1-4be8-ac10-6934928e53d0 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:12.569886 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1165/2299] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:18:12.570165 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae27da5-b82c-4cd3-a977-e055a512b229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:12.570784 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d877b-67e8-4ef5-869d-00d7021f9884 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:12.570910 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae27da5-b82c-4cd3-a977-e055a512b229 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:12.571131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae27da5-b82c-4cd3-a977-e055a512b229 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:12.571468 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae27da5-b82c-4cd3-a977-e055a512b229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:12.580619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d877b-67e8-4ef5-869d-00d7021f9884 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:12.581076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae27da5-b82c-4cd3-a977-e055a512b229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:12.582905 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae27da5-b82c-4cd3-a977-e055a512b229 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:12.585116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683978cb-1e19-419a-9e00-11c439e6f121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:12.586031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683978cb-1e19-419a-9e00-11c439e6f121 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:12.586313 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683978cb-1e19-419a-9e00-11c439e6f121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:12.586581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683978cb-1e19-419a-9e00-11c439e6f121 admin admin] PATH_INFO: `/v3/projects/4e726ed669c64a9b9d618886f1cc96de/users/32b9a1c24fe84019a311b1331ea2c004/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:12.592622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1135/2300] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:18:12.602460 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683978cb-1e19-419a-9e00-11c439e6f121 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e726ed669c64a9b9d618886f1cc96de, user_id=32b9a1c24fe84019a311b1331ea2c004, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:12.605399 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683978cb-1e19-419a-9e00-11c439e6f121 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:12.619327 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1166/2301] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:18:12 2022] PUT /identity/v3/projects/4e726ed669c64a9b9d618886f1cc96de/users/32b9a1c24fe84019a311b1331ea2c004/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:12.624461 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23739e-10ad-4b5a-b724-0aa4e267e6ef None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:12.624461 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23739e-10ad-4b5a-b724-0aa4e267e6ef None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:12.624877 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23739e-10ad-4b5a-b724-0aa4e267e6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:12.643577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-bf23739e-10ad-4b5a-b724-0aa4e267e6ef None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:12.645255 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-bf23739e-10ad-4b5a-b724-0aa4e267e6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:12.721968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1136/2302] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:12.733091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683978cb-1e19-419a-9e00-11c439e6f121 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:12.734916 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-683978cb-1e19-419a-9e00-11c439e6f121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:12.819021 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683978cb-1e19-419a-9e00-11c439e6f121 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:12.823256 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42a9e9-5001-49a7-a3ab-2196a68e2e54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:12.823986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42a9e9-5001-49a7-a3ab-2196a68e2e54 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:12.824317 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42a9e9-5001-49a7-a3ab-2196a68e2e54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:12.824735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42a9e9-5001-49a7-a3ab-2196a68e2e54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:12.834794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42a9e9-5001-49a7-a3ab-2196a68e2e54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:12.836430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42a9e9-5001-49a7-a3ab-2196a68e2e54 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:12.849311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1167/2303] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:12.993173 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23739e-10ad-4b5a-b724-0aa4e267e6ef None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:12.994933 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-bf23739e-10ad-4b5a-b724-0aa4e267e6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:13.008593 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23739e-10ad-4b5a-b724-0aa4e267e6ef None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:13.013390 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04251a5f-49c3-45f3-ab80-72a06ebf8621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:13.014257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04251a5f-49c3-45f3-ab80-72a06ebf8621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:13.014537 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04251a5f-49c3-45f3-ab80-72a06ebf8621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:13.015106 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04251a5f-49c3-45f3-ab80-72a06ebf8621 admin admin] PATH_INFO: `/v3/projects/956b474bf9d249e5ba458f5f04da84eb` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:13.016356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04251a5f-49c3-45f3-ab80-72a06ebf8621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=956b474bf9d249e5ba458f5f04da84eb)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:13.018268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04251a5f-49c3-45f3-ab80-72a06ebf8621 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:13.069658 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-04251a5f-49c3-45f3-ab80-72a06ebf8621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '956b474bf9d249e5ba458f5f04da84eb', 'request_id': 'req-04251a5f-49c3-45f3-ab80-72a06ebf8621'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:13.105060 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-04251a5f-49c3-45f3-ab80-72a06ebf8621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '956b474bf9d249e5ba458f5f04da84eb', 'request_id': 'req-04251a5f-49c3-45f3-ab80-72a06ebf8621'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:13.113314 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-04251a5f-49c3-45f3-ab80-72a06ebf8621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '956b474bf9d249e5ba458f5f04da84eb', 'request_id': 'req-04251a5f-49c3-45f3-ab80-72a06ebf8621'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:13.129306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-04251a5f-49c3-45f3-ab80-72a06ebf8621 admin admin] The token cache is being invalidate because project 956b474bf9d249e5ba458f5f04da84eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:18:13.129814 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-04251a5f-49c3-45f3-ab80-72a06ebf8621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:13.130737 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-04251a5f-49c3-45f3-ab80-72a06ebf8621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:13.132586 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1137/2304] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:18:12 2022] DELETE /identity/v3/projects/956b474bf9d249e5ba458f5f04da84eb => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:13.137530 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca740ba7-6c07-406f-b570-ffd243afcd51 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:13.137820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca740ba7-6c07-406f-b570-ffd243afcd51 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:13.138119 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca740ba7-6c07-406f-b570-ffd243afcd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:13.157789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ca740ba7-6c07-406f-b570-ffd243afcd51 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:13.159879 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ca740ba7-6c07-406f-b570-ffd243afcd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:13.254896 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1168/2305] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:13.263846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a10c4-3b8b-44dc-8d1a-5bd278f12f65 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:13.264518 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a10c4-3b8b-44dc-8d1a-5bd278f12f65 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:13.265684 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a10c4-3b8b-44dc-8d1a-5bd278f12f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:13.295509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a63a10c4-3b8b-44dc-8d1a-5bd278f12f65 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:13.298102 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a63a10c4-3b8b-44dc-8d1a-5bd278f12f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:13.313096 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1138/2306] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:13.317378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca740ba7-6c07-406f-b570-ffd243afcd51 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:13.328304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca740ba7-6c07-406f-b570-ffd243afcd51 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:13.333172 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab082d-1c85-4595-8414-efa38c2038ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:13.333842 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab082d-1c85-4595-8414-efa38c2038ba admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:13.334121 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab082d-1c85-4595-8414-efa38c2038ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:13.334393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab082d-1c85-4595-8414-efa38c2038ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:13.335122 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab082d-1c85-4595-8414-efa38c2038ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:18:13.335428 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab082d-1c85-4595-8414-efa38c2038ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:13.337107 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab082d-1c85-4595-8414-efa38c2038ba admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:13.352709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1169/2307] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:18:13.354213 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a10c4-3b8b-44dc-8d1a-5bd278f12f65 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:13.363565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a10c4-3b8b-44dc-8d1a-5bd278f12f65 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:13.369236 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e23fe-8bfb-41c9-92fe-2d4de9c8d030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:13.370079 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e23fe-8bfb-41c9-92fe-2d4de9c8d030 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:13.370427 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e23fe-8bfb-41c9-92fe-2d4de9c8d030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:13.370973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e23fe-8bfb-41c9-92fe-2d4de9c8d030 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:13.371543 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e23fe-8bfb-41c9-92fe-2d4de9c8d030 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:13.373613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e23fe-8bfb-41c9-92fe-2d4de9c8d030 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:13.378602 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:13.407427 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1139/2308] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:18:13 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:18:13.411360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab082d-1c85-4595-8414-efa38c2038ba admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:13.421539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab082d-1c85-4595-8414-efa38c2038ba admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:13.477581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:13.496913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6d318-b432-41f5-9886-776cffd7b78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:13.497601 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6d318-b432-41f5-9886-776cffd7b78c admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:13.497879 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6d318-b432-41f5-9886-776cffd7b78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:13.498153 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6d318-b432-41f5-9886-776cffd7b78c admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:13.498913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6d318-b432-41f5-9886-776cffd7b78c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:13.500958 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6d318-b432-41f5-9886-776cffd7b78c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:13.527446 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1170/2309] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:18:13 2022] POST /identity/v3/users => generated 285 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:18:13.531956 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e23fe-8bfb-41c9-92fe-2d4de9c8d030 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:13.543043 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e23fe-8bfb-41c9-92fe-2d4de9c8d030 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:13.548207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228491d-5c48-420f-8944-7960a332e077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:13.548882 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228491d-5c48-420f-8944-7960a332e077 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:13.549162 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228491d-5c48-420f-8944-7960a332e077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:13.549436 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228491d-5c48-420f-8944-7960a332e077 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:13.550170 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228491d-5c48-420f-8944-7960a332e077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:13.551825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228491d-5c48-420f-8944-7960a332e077 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:13.565370 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1140/2310] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:18:13.571706 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6d318-b432-41f5-9886-776cffd7b78c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:13.584184 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6d318-b432-41f5-9886-776cffd7b78c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:13.593143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87362727-c492-4a9c-b43a-99325d8166b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:13.594249 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87362727-c492-4a9c-b43a-99325d8166b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:13.594625 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87362727-c492-4a9c-b43a-99325d8166b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:13.595114 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87362727-c492-4a9c-b43a-99325d8166b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:13.596798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87362727-c492-4a9c-b43a-99325d8166b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:13.598463 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87362727-c492-4a9c-b43a-99325d8166b6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:13.630356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1171/2311] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:18:13.635421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228491d-5c48-420f-8944-7960a332e077 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:13.651117 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228491d-5c48-420f-8944-7960a332e077 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:13.658577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15bf50-9f67-4bd6-b7ae-a5dd4a46aebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:13.659587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15bf50-9f67-4bd6-b7ae-a5dd4a46aebd admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:13.659989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15bf50-9f67-4bd6-b7ae-a5dd4a46aebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:13.660318 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15bf50-9f67-4bd6-b7ae-a5dd4a46aebd admin admin] PATH_INFO: `/v3/projects/17dea7e9f26c4b74a10d5e00a395fbd7/users/4f9b08c0c7fd4d2e957afcefca500751/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:13.695502 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15bf50-9f67-4bd6-b7ae-a5dd4a46aebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=17dea7e9f26c4b74a10d5e00a395fbd7, user_id=4f9b08c0c7fd4d2e957afcefca500751, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:13.700698 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15bf50-9f67-4bd6-b7ae-a5dd4a46aebd admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:13.715780 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1141/2312] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:18:13 2022] PUT /identity/v3/projects/17dea7e9f26c4b74a10d5e00a395fbd7/users/4f9b08c0c7fd4d2e957afcefca500751/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:18:13.723129 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69e5c0-320d-4f5b-bcc2-95947a757f2c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:13.723689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69e5c0-320d-4f5b-bcc2-95947a757f2c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:13.724181 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69e5c0-320d-4f5b-bcc2-95947a757f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:13.750704 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-bb69e5c0-320d-4f5b-bcc2-95947a757f2c None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:13.752131 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bb69e5c0-320d-4f5b-bcc2-95947a757f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:13.845317 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1172/2313] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:18:13.857477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15bf50-9f67-4bd6-b7ae-a5dd4a46aebd admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:13.859229 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-aa15bf50-9f67-4bd6-b7ae-a5dd4a46aebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:13.979721 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15bf50-9f67-4bd6-b7ae-a5dd4a46aebd admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:13.991527 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999904b-fbd6-4656-84b2-e2976c1924f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:14.000885 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eee357-ed2f-4df5-9016-998f9c8e8b64 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:14.001247 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eee357-ed2f-4df5-9016-998f9c8e8b64 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:14.001650 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eee357-ed2f-4df5-9016-998f9c8e8b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:14.003783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999904b-fbd6-4656-84b2-e2976c1924f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:14.004284 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999904b-fbd6-4656-84b2-e2976c1924f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:14.004575 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999904b-fbd6-4656-84b2-e2976c1924f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:14.022747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d0eee357-ed2f-4df5-9016-998f9c8e8b64 None None] MFA Rules not processed for user `e802da0b078e457da466a9a7b0f15085`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:14.024090 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d0eee357-ed2f-4df5-9016-998f9c8e8b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:14.050780 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999904b-fbd6-4656-84b2-e2976c1924f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:14.058596 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999904b-fbd6-4656-84b2-e2976c1924f3 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:14.091561 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1142/2314] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:14.093116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1173/2315] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Jun 27 21:18:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:18:14.105986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999904b-fbd6-4656-84b2-e2976c1924f3 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:14.108076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4999904b-fbd6-4656-84b2-e2976c1924f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:14.238317 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999904b-fbd6-4656-84b2-e2976c1924f3 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:14.248118 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b684911-af76-4b26-b9af-d8a50b34d6ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28acb0322114aebafcf62bdf9a346df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:14.249335 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b684911-af76-4b26-b9af-d8a50b34d6ab service swift] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:14.249805 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b684911-af76-4b26-b9af-d8a50b34d6ab service swift] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:14.250425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b684911-af76-4b26-b9af-d8a50b34d6ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:14.263227 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b684911-af76-4b26-b9af-d8a50b34d6ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:14.265763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b684911-af76-4b26-b9af-d8a50b34d6ab service swift] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:14.278604 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1143/2316] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Jun 27 21:18:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:18:22.816471 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f20e6-4c79-46c7-be27-e39976009f7a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:22.817058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f20e6-4c79-46c7-be27-e39976009f7a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:22.817058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f20e6-4c79-46c7-be27-e39976009f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:22.867498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-954f20e6-4c79-46c7-be27-e39976009f7a None None] MFA Rules not processed for user `32b9a1c24fe84019a311b1331ea2c004`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:22.869550 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-954f20e6-4c79-46c7-be27-e39976009f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:22.986585 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:23.002444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1174/2317] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:23.006842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b684911-af76-4b26-b9af-d8a50b34d6ab service swift] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:23.008436 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7b684911-af76-4b26-b9af-d8a50b34d6ab service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:23.045349 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b684911-af76-4b26-b9af-d8a50b34d6ab service swift] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:23.051443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86742b9-a5e3-4361-9c83-b8e58711a1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:23.052217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86742b9-a5e3-4361-9c83-b8e58711a1e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:23.052500 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86742b9-a5e3-4361-9c83-b8e58711a1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:23.052787 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86742b9-a5e3-4361-9c83-b8e58711a1e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:23.054993 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86742b9-a5e3-4361-9c83-b8e58711a1e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:23.064205 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86742b9-a5e3-4361-9c83-b8e58711a1e5 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:23.138485 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954f20e6-4c79-46c7-be27-e39976009f7a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:23.140814 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1144/2318] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:18:23 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:18:23.160133 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954f20e6-4c79-46c7-be27-e39976009f7a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:23.249981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0caf5-28cf-48fc-bff2-461b78d48414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:23.252261 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0caf5-28cf-48fc-bff2-461b78d48414 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:23.252816 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0caf5-28cf-48fc-bff2-461b78d48414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:23.253318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0caf5-28cf-48fc-bff2-461b78d48414 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:23.254195 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0caf5-28cf-48fc-bff2-461b78d48414 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:23.256409 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0caf5-28cf-48fc-bff2-461b78d48414 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:23.295312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1175/2319] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:18:23 2022] POST /identity/v3/users => generated 295 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:18:23.298072 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86742b9-a5e3-4361-9c83-b8e58711a1e5 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:23.314526 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86742b9-a5e3-4361-9c83-b8e58711a1e5 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:23.321154 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a32032b-eeb5-44ce-83f5-ba6d5f033d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:23.322479 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32032b-eeb5-44ce-83f5-ba6d5f033d7f admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:23.323411 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32032b-eeb5-44ce-83f5-ba6d5f033d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:23.323963 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32032b-eeb5-44ce-83f5-ba6d5f033d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:23.325862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a32032b-eeb5-44ce-83f5-ba6d5f033d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:23.329264 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a32032b-eeb5-44ce-83f5-ba6d5f033d7f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:23.350864 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1145/2320] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:18:23.355312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0caf5-28cf-48fc-bff2-461b78d48414 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:23.375903 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0caf5-28cf-48fc-bff2-461b78d48414 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:23.384640 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e6333-8298-4f2a-bdfc-d48b38ee2643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:23.386335 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e6333-8298-4f2a-bdfc-d48b38ee2643 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:23.386887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e6333-8298-4f2a-bdfc-d48b38ee2643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:23.387329 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e6333-8298-4f2a-bdfc-d48b38ee2643 admin admin] PATH_INFO: `/v3/projects/658a622dbd9a48fbb82208c50ab930d3/users/0e8a1b34b723424e8552493a6c582aa6/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:23.417998 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e6333-8298-4f2a-bdfc-d48b38ee2643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=658a622dbd9a48fbb82208c50ab930d3, user_id=0e8a1b34b723424e8552493a6c582aa6, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:23.422500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e6333-8298-4f2a-bdfc-d48b38ee2643 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:23.456715 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1176/2321] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:18:23 2022] PUT /identity/v3/projects/658a622dbd9a48fbb82208c50ab930d3/users/0e8a1b34b723424e8552493a6c582aa6/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:23.460891 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a32032b-eeb5-44ce-83f5-ba6d5f033d7f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:23.475255 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a32032b-eeb5-44ce-83f5-ba6d5f033d7f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:23.497892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:23.649794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fdab5-2879-424e-b2d3-853fdaf489ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:23.650493 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fdab5-2879-424e-b2d3-853fdaf489ed admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:23.650775 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fdab5-2879-424e-b2d3-853fdaf489ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:23.651044 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fdab5-2879-424e-b2d3-853fdaf489ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:23.651994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fdab5-2879-424e-b2d3-853fdaf489ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:23.654005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fdab5-2879-424e-b2d3-853fdaf489ed admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:23.685851 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e6333-8298-4f2a-bdfc-d48b38ee2643 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:23.688372 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1146/2322] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:18:23.709293 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e6333-8298-4f2a-bdfc-d48b38ee2643 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:23.715023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40046747-23a7-47b8-b139-06ba965e5948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:23.716065 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40046747-23a7-47b8-b139-06ba965e5948 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:23.716348 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40046747-23a7-47b8-b139-06ba965e5948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:23.716624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40046747-23a7-47b8-b139-06ba965e5948 admin admin] PATH_INFO: `/v3/projects/658a622dbd9a48fbb82208c50ab930d3/users/0e8a1b34b723424e8552493a6c582aa6/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:23.718689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40046747-23a7-47b8-b139-06ba965e5948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=658a622dbd9a48fbb82208c50ab930d3, user_id=0e8a1b34b723424e8552493a6c582aa6, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:23.721440 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40046747-23a7-47b8-b139-06ba965e5948 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:23.751680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1177/2323] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:18:23 2022] PUT /identity/v3/projects/658a622dbd9a48fbb82208c50ab930d3/users/0e8a1b34b723424e8552493a6c582aa6/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:18:26.134920 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450e5ac-9b7d-4b71-b00e-28fe6e4c5d33 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:26.135455 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450e5ac-9b7d-4b71-b00e-28fe6e4c5d33 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:26.135548 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450e5ac-9b7d-4b71-b00e-28fe6e4c5d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:26.202650 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-0450e5ac-9b7d-4b71-b00e-28fe6e4c5d33 None None] MFA Rules not processed for user `4f9b08c0c7fd4d2e957afcefca500751`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:26.204756 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0450e5ac-9b7d-4b71-b00e-28fe6e4c5d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:26.361608 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1147/2324] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:26.362485 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40046747-23a7-47b8-b139-06ba965e5948 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:26.516486 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40046747-23a7-47b8-b139-06ba965e5948 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:26.523141 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ad43b-b907-42ec-8de0-6292200e60fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:26.524178 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ad43b-b907-42ec-8de0-6292200e60fe admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:26.524643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ad43b-b907-42ec-8de0-6292200e60fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:26.525453 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ad43b-b907-42ec-8de0-6292200e60fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:26.526370 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ad43b-b907-42ec-8de0-6292200e60fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:26.528685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ad43b-b907-42ec-8de0-6292200e60fe admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:26.592617 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1178/2325] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:18:26 2022] POST /identity/v3/projects => generated 373 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:18:26.597570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450e5ac-9b7d-4b71-b00e-28fe6e4c5d33 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:26.614810 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450e5ac-9b7d-4b71-b00e-28fe6e4c5d33 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:26.778691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa8d78-f3ce-4557-9e5c-a1fe2ae47a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:26.779637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa8d78-f3ce-4557-9e5c-a1fe2ae47a77 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:26.780075 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa8d78-f3ce-4557-9e5c-a1fe2ae47a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:26.780504 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa8d78-f3ce-4557-9e5c-a1fe2ae47a77 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:26.781429 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa8d78-f3ce-4557-9e5c-a1fe2ae47a77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:26.787668 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa8d78-f3ce-4557-9e5c-a1fe2ae47a77 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:26.831676 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1148/2326] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:18:26 2022] POST /identity/v3/users => generated 293 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:18:26.835683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ad43b-b907-42ec-8de0-6292200e60fe admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:26.852558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ad43b-b907-42ec-8de0-6292200e60fe admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:26.864914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50cd69-500e-4bf2-8e4d-7ea5ad29d58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:26.865764 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50cd69-500e-4bf2-8e4d-7ea5ad29d58e admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:26.866141 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50cd69-500e-4bf2-8e4d-7ea5ad29d58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:26.866510 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50cd69-500e-4bf2-8e4d-7ea5ad29d58e admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:26.868296 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50cd69-500e-4bf2-8e4d-7ea5ad29d58e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:26.873462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50cd69-500e-4bf2-8e4d-7ea5ad29d58e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:26.905007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1179/2327] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:18:26.909208 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa8d78-f3ce-4557-9e5c-a1fe2ae47a77 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:26.928722 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa8d78-f3ce-4557-9e5c-a1fe2ae47a77 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:26.939753 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c293576-59c1-47bc-a36d-40aa09263dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:26.942062 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c293576-59c1-47bc-a36d-40aa09263dda admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:26.942763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c293576-59c1-47bc-a36d-40aa09263dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:26.942996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c293576-59c1-47bc-a36d-40aa09263dda admin admin] PATH_INFO: `/v3/projects/4370c77bae0e4c4f9f6772a7d66cdab3/users/eb045f8f97304e9387d29ea6f6b20ef3/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:26.980637 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c293576-59c1-47bc-a36d-40aa09263dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=4370c77bae0e4c4f9f6772a7d66cdab3, user_id=eb045f8f97304e9387d29ea6f6b20ef3, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:26.983118 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c293576-59c1-47bc-a36d-40aa09263dda admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:27.018716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1149/2328] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:18:26 2022] PUT /identity/v3/projects/4370c77bae0e4c4f9f6772a7d66cdab3/users/eb045f8f97304e9387d29ea6f6b20ef3/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:18:27.022859 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50cd69-500e-4bf2-8e4d-7ea5ad29d58e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:27.039430 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50cd69-500e-4bf2-8e4d-7ea5ad29d58e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:27.103286 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:27.195493 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01081542-5273-42bc-9584-673060389317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:27.196657 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01081542-5273-42bc-9584-673060389317 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:27.197037 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01081542-5273-42bc-9584-673060389317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:27.197417 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01081542-5273-42bc-9584-673060389317 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:27.198266 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01081542-5273-42bc-9584-673060389317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:27.200092 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01081542-5273-42bc-9584-673060389317 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:27.240239 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1180/2329] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:18:27.244561 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c293576-59c1-47bc-a36d-40aa09263dda admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:27.253987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:27.285779 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c293576-59c1-47bc-a36d-40aa09263dda admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:27.308987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8f2b7-bf28-4f4d-8171-8f1935997cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:27.314923 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8f2b7-bf28-4f4d-8171-8f1935997cef admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:27.317114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8f2b7-bf28-4f4d-8171-8f1935997cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:27.320452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8f2b7-bf28-4f4d-8171-8f1935997cef admin admin] PATH_INFO: `/v3/projects/4370c77bae0e4c4f9f6772a7d66cdab3/users/eb045f8f97304e9387d29ea6f6b20ef3/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:27.327186 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8f2b7-bf28-4f4d-8171-8f1935997cef admin admin] RBAC: Authorizing `identity:create_grant(project_id=4370c77bae0e4c4f9f6772a7d66cdab3, user_id=eb045f8f97304e9387d29ea6f6b20ef3, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:27.338170 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8f2b7-bf28-4f4d-8171-8f1935997cef admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:27.379838 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1150/2330] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:18:27 2022] PUT /identity/v3/projects/4370c77bae0e4c4f9f6772a7d66cdab3/users/eb045f8f97304e9387d29ea6f6b20ef3/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:37.092971 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd4548-4433-4766-ac4c-46680ac690d2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:37.093515 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd4548-4433-4766-ac4c-46680ac690d2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:37.093751 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd4548-4433-4766-ac4c-46680ac690d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:37.225412 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-efcd4548-4433-4766-ac4c-46680ac690d2 None None] MFA Rules not processed for user `0e8a1b34b723424e8552493a6c582aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:37.231390 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-efcd4548-4433-4766-ac4c-46680ac690d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:37.368536 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=4,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:37.543196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1181/2331] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 451 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:37.544067 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8f2b7-bf28-4f4d-8171-8f1935997cef admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:37.546061 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8fb8f2b7-bf28-4f4d-8171-8f1935997cef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:37.749390 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8f2b7-bf28-4f4d-8171-8f1935997cef admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:37.763471 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0730ec-bf4c-4ad1-acf1-e36948395596 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:37.771788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0730ec-bf4c-4ad1-acf1-e36948395596 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:37.772477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0730ec-bf4c-4ad1-acf1-e36948395596 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:37.772908 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0730ec-bf4c-4ad1-acf1-e36948395596 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:37.861615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0730ec-bf4c-4ad1-acf1-e36948395596 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:37.868816 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0730ec-bf4c-4ad1-acf1-e36948395596 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:37.895977 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1151/2332] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:18:40.531795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6b693-a9f4-4fda-91b2-0357a8bd9760 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:40.532271 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6b693-a9f4-4fda-91b2-0357a8bd9760 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:40.532605 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6b693-a9f4-4fda-91b2-0357a8bd9760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:40.560621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6ef6b693-a9f4-4fda-91b2-0357a8bd9760 None None] MFA Rules not processed for user `eb045f8f97304e9387d29ea6f6b20ef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:40.561771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6ef6b693-a9f4-4fda-91b2-0357a8bd9760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:40.626072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:40.631670 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1182/2333] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:40.645974 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0730ec-bf4c-4ad1-acf1-e36948395596 service nova] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:40.658184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0730ec-bf4c-4ad1-acf1-e36948395596 service nova] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:40.666500 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077cc8a-689c-43ea-a458-00b91f034a91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:40.667306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077cc8a-689c-43ea-a458-00b91f034a91 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:40.667693 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077cc8a-689c-43ea-a458-00b91f034a91 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:40.668121 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077cc8a-689c-43ea-a458-00b91f034a91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:40.731594 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:40.734295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077cc8a-689c-43ea-a458-00b91f034a91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:40.748948 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077cc8a-689c-43ea-a458-00b91f034a91 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:40.761421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1152/2334] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:40 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:42.069052 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0260b3-844b-4070-8195-29f4b8388fad None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:42.069588 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0260b3-844b-4070-8195-29f4b8388fad None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:42.069686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0260b3-844b-4070-8195-29f4b8388fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:42.092670 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8f0260b3-844b-4070-8195-29f4b8388fad None None] MFA Rules not processed for user `4f9b08c0c7fd4d2e957afcefca500751`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:42.094197 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8f0260b3-844b-4070-8195-29f4b8388fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:42.102532 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1183/2335] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:42 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:43.917479 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077cc8a-689c-43ea-a458-00b91f034a91 service nova] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:43.918997 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8077cc8a-689c-43ea-a458-00b91f034a91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:44.147193 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077cc8a-689c-43ea-a458-00b91f034a91 service nova] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:44.153063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddafa82-b21f-435d-aa3f-acbb0e1dad71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:44.154630 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddafa82-b21f-435d-aa3f-acbb0e1dad71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:44.154975 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddafa82-b21f-435d-aa3f-acbb0e1dad71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:44.156055 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddafa82-b21f-435d-aa3f-acbb0e1dad71 admin admin] PATH_INFO: `/v3/users/0c2e0d2445714a568580da468ce7d087` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:44.157538 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddafa82-b21f-435d-aa3f-acbb0e1dad71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c2e0d2445714a568580da468ce7d087)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:44.160146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddafa82-b21f-435d-aa3f-acbb0e1dad71 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:44.255197 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4ddafa82-b21f-435d-aa3f-acbb0e1dad71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c2e0d2445714a568580da468ce7d087'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:44.261477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4ddafa82-b21f-435d-aa3f-acbb0e1dad71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c2e0d2445714a568580da468ce7d087'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:44.263703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4ddafa82-b21f-435d-aa3f-acbb0e1dad71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c2e0d2445714a568580da468ce7d087'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:44.309011 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4ddafa82-b21f-435d-aa3f-acbb0e1dad71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c2e0d2445714a568580da468ce7d087'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:44.341875 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4ddafa82-b21f-435d-aa3f-acbb0e1dad71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c2e0d2445714a568580da468ce7d087'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:44.369418 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1153/2336] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:18:43 2022] DELETE /identity/v3/users/0c2e0d2445714a568580da468ce7d087 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:44.373968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ae786-4ae7-4b81-a122-afda6d8ccaa6 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:44.374491 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ae786-4ae7-4b81-a122-afda6d8ccaa6 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:44.374612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ae786-4ae7-4b81-a122-afda6d8ccaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:44.463990 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a68ae786-4ae7-4b81-a122-afda6d8ccaa6 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:44.465933 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a68ae786-4ae7-4b81-a122-afda6d8ccaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:44.593949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1184/2337] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:44.605955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddafa82-b21f-435d-aa3f-acbb0e1dad71 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:44.619258 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4ddafa82-b21f-435d-aa3f-acbb0e1dad71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:44.699431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:44.748293 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c0880-0d6b-44a8-ae3a-3be0c4450ba8 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:44.749305 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c0880-0d6b-44a8-ae3a-3be0c4450ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:44.749742 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c0880-0d6b-44a8-ae3a-3be0c4450ba8 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:44.756406 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1185/2338] 10.222.0.27 () {54 vars in 921 bytes} [Mon Jun 27 21:18:44 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:18:44.766971 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9d242-8acd-4334-a994-7293968bbac4 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:44.767506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9d242-8acd-4334-a994-7293968bbac4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:44.768035 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9d242-8acd-4334-a994-7293968bbac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:44.770769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddafa82-b21f-435d-aa3f-acbb0e1dad71 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:44.783888 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9563e-9c54-4b78-832a-eb1f201d360c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:44.784568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9563e-9c54-4b78-832a-eb1f201d360c service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:44.784849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9563e-9c54-4b78-832a-eb1f201d360c service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:44.785123 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9563e-9c54-4b78-832a-eb1f201d360c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:44.806699 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-50d9d242-8acd-4334-a994-7293968bbac4 None None] MFA Rules not processed for user `8d7fe01ef64f46f6a0e77d5945219b33`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:44.808953 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-50d9d242-8acd-4334-a994-7293968bbac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:44.809441 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c9563e-9c54-4b78-832a-eb1f201d360c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:44.814407 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c9563e-9c54-4b78-832a-eb1f201d360c service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:44.837908 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1154/2339] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:44.946957 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1186/2340] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Jun 27 21:18:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:45.268595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9563e-9c54-4b78-832a-eb1f201d360c service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:45.270158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d1c9563e-9c54-4b78-832a-eb1f201d360c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:45.288963 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9563e-9c54-4b78-832a-eb1f201d360c service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:45.298719 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ed00d-0898-4c52-8ad6-48d67c580dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:45.299532 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ed00d-0898-4c52-8ad6-48d67c580dbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:45.300145 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ed00d-0898-4c52-8ad6-48d67c580dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:45.300269 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ed00d-0898-4c52-8ad6-48d67c580dbd admin admin] PATH_INFO: `/v3/projects/0f10581267df454c93cf3554fb633e3d` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:45.301786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ed00d-0898-4c52-8ad6-48d67c580dbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f10581267df454c93cf3554fb633e3d)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:45.303868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ed00d-0898-4c52-8ad6-48d67c580dbd admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:45.363136 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b1ed00d-0898-4c52-8ad6-48d67c580dbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f10581267df454c93cf3554fb633e3d', 'request_id': 'req-9b1ed00d-0898-4c52-8ad6-48d67c580dbd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:45.371672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b1ed00d-0898-4c52-8ad6-48d67c580dbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f10581267df454c93cf3554fb633e3d', 'request_id': 'req-9b1ed00d-0898-4c52-8ad6-48d67c580dbd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:45.378086 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b1ed00d-0898-4c52-8ad6-48d67c580dbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f10581267df454c93cf3554fb633e3d', 'request_id': 'req-9b1ed00d-0898-4c52-8ad6-48d67c580dbd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:45.392247 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b1ed00d-0898-4c52-8ad6-48d67c580dbd admin admin] The token cache is being invalidate because project 0f10581267df454c93cf3554fb633e3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:18:45.392550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b1ed00d-0898-4c52-8ad6-48d67c580dbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:45.393313 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b1ed00d-0898-4c52-8ad6-48d67c580dbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:45.395420 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1155/2341] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:18:45 2022] DELETE /identity/v3/projects/0f10581267df454c93cf3554fb633e3d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:45.401190 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29db6f-e7d1-441c-810e-3061bd476436 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:45.401735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29db6f-e7d1-441c-810e-3061bd476436 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:45.401735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29db6f-e7d1-441c-810e-3061bd476436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:45.443485 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-fe29db6f-e7d1-441c-810e-3061bd476436 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:45.444756 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fe29db6f-e7d1-441c-810e-3061bd476436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:45.517507 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1187/2342] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:18:45.526301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35904d8-cb11-407c-9d2e-8d79130c1f8a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:45.526655 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35904d8-cb11-407c-9d2e-8d79130c1f8a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:45.527017 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35904d8-cb11-407c-9d2e-8d79130c1f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:45.547406 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e35904d8-cb11-407c-9d2e-8d79130c1f8a None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:45.548946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e35904d8-cb11-407c-9d2e-8d79130c1f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:45.562741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1156/2343] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:18:45.567746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29db6f-e7d1-441c-810e-3061bd476436 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:45.573292 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:45.580062 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29db6f-e7d1-441c-810e-3061bd476436 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:45.585288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1a426-e123-4bb2-b0cb-8f212cdb8913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:45.586084 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1a426-e123-4bb2-b0cb-8f212cdb8913 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:45.586541 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1a426-e123-4bb2-b0cb-8f212cdb8913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:45.586991 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1a426-e123-4bb2-b0cb-8f212cdb8913 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:45.587780 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1a426-e123-4bb2-b0cb-8f212cdb8913 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:18:45.588092 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1a426-e123-4bb2-b0cb-8f212cdb8913 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:45.589794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1a426-e123-4bb2-b0cb-8f212cdb8913 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:45.604751 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1188/2344] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:18:45.609472 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35904d8-cb11-407c-9d2e-8d79130c1f8a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:45.621430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35904d8-cb11-407c-9d2e-8d79130c1f8a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:45.627004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec3eb5-fa4d-4253-b6d4-8e75cf7437cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:45.627274 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec3eb5-fa4d-4253-b6d4-8e75cf7437cf admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:45.627649 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec3eb5-fa4d-4253-b6d4-8e75cf7437cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:45.627938 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec3eb5-fa4d-4253-b6d4-8e75cf7437cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:45.628640 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec3eb5-fa4d-4253-b6d4-8e75cf7437cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:45.630528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec3eb5-fa4d-4253-b6d4-8e75cf7437cf admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:45.665003 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1157/2345] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:18:45 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:18:45.669476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1a426-e123-4bb2-b0cb-8f212cdb8913 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:45.680491 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1a426-e123-4bb2-b0cb-8f212cdb8913 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:45.761149 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feecf21-4b2d-4182-bbdc-10af1236c6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:45.761541 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feecf21-4b2d-4182-bbdc-10af1236c6d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:45.761850 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feecf21-4b2d-4182-bbdc-10af1236c6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:45.762093 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feecf21-4b2d-4182-bbdc-10af1236c6d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:45.763011 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feecf21-4b2d-4182-bbdc-10af1236c6d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:45.765378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feecf21-4b2d-4182-bbdc-10af1236c6d5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:45.796422 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1189/2346] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:18:45 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:18:45.800702 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec3eb5-fa4d-4253-b6d4-8e75cf7437cf admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:45.813902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec3eb5-fa4d-4253-b6d4-8e75cf7437cf admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:45.818885 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb530c6c-1b7d-4c94-b218-bec6df3ec7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:45.819719 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb530c6c-1b7d-4c94-b218-bec6df3ec7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:45.820265 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb530c6c-1b7d-4c94-b218-bec6df3ec7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:45.820639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb530c6c-1b7d-4c94-b218-bec6df3ec7c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:45.821455 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb530c6c-1b7d-4c94-b218-bec6df3ec7c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:45.823185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb530c6c-1b7d-4c94-b218-bec6df3ec7c3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:45.835600 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1158/2347] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:18:45.845001 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feecf21-4b2d-4182-bbdc-10af1236c6d5 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:45.856029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feecf21-4b2d-4182-bbdc-10af1236c6d5 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:45.860757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e0c7f-03b6-4aca-bfdb-f600dcb29182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:45.861410 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e0c7f-03b6-4aca-bfdb-f600dcb29182 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:45.861718 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e0c7f-03b6-4aca-bfdb-f600dcb29182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:45.861980 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e0c7f-03b6-4aca-bfdb-f600dcb29182 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:45.862709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e0c7f-03b6-4aca-bfdb-f600dcb29182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:45.864368 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e0c7f-03b6-4aca-bfdb-f600dcb29182 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:45.876344 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1190/2348] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:18:45.881426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb530c6c-1b7d-4c94-b218-bec6df3ec7c3 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:45.893469 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb530c6c-1b7d-4c94-b218-bec6df3ec7c3 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:45.898772 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def63c6e-ca04-4078-b8f3-6212cd003097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:45.899699 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def63c6e-ca04-4078-b8f3-6212cd003097 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:45.899979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def63c6e-ca04-4078-b8f3-6212cd003097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:45.900251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def63c6e-ca04-4078-b8f3-6212cd003097 admin admin] PATH_INFO: `/v3/projects/737e875f4cb54213bc1d2cb71a3684b4/users/d5d3d2c32d5f48bda3eb0f4c42f2bd89/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:45.915069 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def63c6e-ca04-4078-b8f3-6212cd003097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=737e875f4cb54213bc1d2cb71a3684b4, user_id=d5d3d2c32d5f48bda3eb0f4c42f2bd89, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:45.917773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def63c6e-ca04-4078-b8f3-6212cd003097 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:45.935223 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1159/2349] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:18:45 2022] PUT /identity/v3/projects/737e875f4cb54213bc1d2cb71a3684b4/users/d5d3d2c32d5f48bda3eb0f4c42f2bd89/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:45.948553 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d89ba-69b8-4594-8a25-ce3b1724b5fc None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:45.948844 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d89ba-69b8-4594-8a25-ce3b1724b5fc None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:45.949122 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d89ba-69b8-4594-8a25-ce3b1724b5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:45.976610 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-189d89ba-69b8-4594-8a25-ce3b1724b5fc None None] MFA Rules not processed for user `d5d3d2c32d5f48bda3eb0f4c42f2bd89`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:45.978075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-189d89ba-69b8-4594-8a25-ce3b1724b5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:46.059443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1191/2350] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:46.070181 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def63c6e-ca04-4078-b8f3-6212cd003097 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:46.071961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-def63c6e-ca04-4078-b8f3-6212cd003097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:46.130849 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:46.135388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def63c6e-ca04-4078-b8f3-6212cd003097 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:46.142019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d1f28-364b-4ab2-8a06-304b4caec67b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:46.142690 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d1f28-364b-4ab2-8a06-304b4caec67b service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:46.142971 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d1f28-364b-4ab2-8a06-304b4caec67b service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:46.143919 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d1f28-364b-4ab2-8a06-304b4caec67b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:46.153938 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d1f28-364b-4ab2-8a06-304b4caec67b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:46.155373 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d1f28-364b-4ab2-8a06-304b4caec67b service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:46.164629 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1160/2351] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:56.056629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:56.163357 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:56.406233 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d89ba-69b8-4594-8a25-ce3b1724b5fc None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:56.476208 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d89ba-69b8-4594-8a25-ce3b1724b5fc None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:56.481044 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131c350-7839-41e9-b4d0-10e2ffd97053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:56.481936 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131c350-7839-41e9-b4d0-10e2ffd97053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:56.482212 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131c350-7839-41e9-b4d0-10e2ffd97053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:56.482487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131c350-7839-41e9-b4d0-10e2ffd97053 admin admin] PATH_INFO: `/v3/users/d5d3d2c32d5f48bda3eb0f4c42f2bd89` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:56.483770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3131c350-7839-41e9-b4d0-10e2ffd97053 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5d3d2c32d5f48bda3eb0f4c42f2bd89)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:56.485810 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3131c350-7839-41e9-b4d0-10e2ffd97053 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:56.542816 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3131c350-7839-41e9-b4d0-10e2ffd97053 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5d3d2c32d5f48bda3eb0f4c42f2bd89'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:56.543473 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3131c350-7839-41e9-b4d0-10e2ffd97053 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5d3d2c32d5f48bda3eb0f4c42f2bd89'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:56.544158 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3131c350-7839-41e9-b4d0-10e2ffd97053 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5d3d2c32d5f48bda3eb0f4c42f2bd89'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:56.566494 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3131c350-7839-41e9-b4d0-10e2ffd97053 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5d3d2c32d5f48bda3eb0f4c42f2bd89'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:56.582048 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3131c350-7839-41e9-b4d0-10e2ffd97053 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5d3d2c32d5f48bda3eb0f4c42f2bd89'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:56.598364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1192/2352] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:18:56 2022] DELETE /identity/v3/users/d5d3d2c32d5f48bda3eb0f4c42f2bd89 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:56.602876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177708f-c451-47a0-978a-58362e06a022 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:56.603160 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177708f-c451-47a0-978a-58362e06a022 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:56.603435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177708f-c451-47a0-978a-58362e06a022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:56.621068 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5177708f-c451-47a0-978a-58362e06a022 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:56.622273 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5177708f-c451-47a0-978a-58362e06a022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:56.685348 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1161/2353] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:56.696719 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131c350-7839-41e9-b4d0-10e2ffd97053 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:56.699030 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3131c350-7839-41e9-b4d0-10e2ffd97053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:56.765600 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131c350-7839-41e9-b4d0-10e2ffd97053 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:56.769337 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525139a-e284-483b-8f6c-b889293f7115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:56.770063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525139a-e284-483b-8f6c-b889293f7115 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:56.770404 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525139a-e284-483b-8f6c-b889293f7115 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:56.770679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525139a-e284-483b-8f6c-b889293f7115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:56.782009 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9525139a-e284-483b-8f6c-b889293f7115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:56.783716 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9525139a-e284-483b-8f6c-b889293f7115 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:56.793801 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1193/2354] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:57.073191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177708f-c451-47a0-978a-58362e06a022 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:57.075222 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5177708f-c451-47a0-978a-58362e06a022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:57.087418 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177708f-c451-47a0-978a-58362e06a022 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:57.092433 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73a18b-180e-4717-b97a-7fbe165101d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:57.093307 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73a18b-180e-4717-b97a-7fbe165101d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:57.093584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73a18b-180e-4717-b97a-7fbe165101d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:57.093877 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73a18b-180e-4717-b97a-7fbe165101d9 admin admin] PATH_INFO: `/v3/projects/737e875f4cb54213bc1d2cb71a3684b4` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:57.095049 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73a18b-180e-4717-b97a-7fbe165101d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=737e875f4cb54213bc1d2cb71a3684b4)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:57.097003 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73a18b-180e-4717-b97a-7fbe165101d9 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:57.146380 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2a73a18b-180e-4717-b97a-7fbe165101d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '737e875f4cb54213bc1d2cb71a3684b4', 'request_id': 'req-2a73a18b-180e-4717-b97a-7fbe165101d9'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:57.154551 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2a73a18b-180e-4717-b97a-7fbe165101d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '737e875f4cb54213bc1d2cb71a3684b4', 'request_id': 'req-2a73a18b-180e-4717-b97a-7fbe165101d9'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:57.160448 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2a73a18b-180e-4717-b97a-7fbe165101d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '737e875f4cb54213bc1d2cb71a3684b4', 'request_id': 'req-2a73a18b-180e-4717-b97a-7fbe165101d9'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:57.172050 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2a73a18b-180e-4717-b97a-7fbe165101d9 admin admin] The token cache is being invalidate because project 737e875f4cb54213bc1d2cb71a3684b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:18:57.172629 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2a73a18b-180e-4717-b97a-7fbe165101d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:57.173037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2a73a18b-180e-4717-b97a-7fbe165101d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:57.174746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1162/2355] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:18:57 2022] DELETE /identity/v3/projects/737e875f4cb54213bc1d2cb71a3684b4 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:57.179695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5594f79-cbeb-426e-b922-7e71cdb037ec None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:57.180117 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5594f79-cbeb-426e-b922-7e71cdb037ec None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:57.180271 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5594f79-cbeb-426e-b922-7e71cdb037ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:57.217502 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b5594f79-cbeb-426e-b922-7e71cdb037ec None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:57.218740 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b5594f79-cbeb-426e-b922-7e71cdb037ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:57.228912 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:57.286167 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1194/2356] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:57.294139 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c630b3c-5a4b-4c87-8939-a08497dc8c8e None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:57.294457 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c630b3c-5a4b-4c87-8939-a08497dc8c8e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:57.294732 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c630b3c-5a4b-4c87-8939-a08497dc8c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:57.313912 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5c630b3c-5a4b-4c87-8939-a08497dc8c8e None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:57.315125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5c630b3c-5a4b-4c87-8939-a08497dc8c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:57.323690 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1163/2357] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:57.328025 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5594f79-cbeb-426e-b922-7e71cdb037ec None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:57.337816 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5594f79-cbeb-426e-b922-7e71cdb037ec None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:57.342739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271efaa-27cd-4551-96a0-2a85e92fb5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:57.343397 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271efaa-27cd-4551-96a0-2a85e92fb5d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:57.343701 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271efaa-27cd-4551-96a0-2a85e92fb5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:57.343973 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271efaa-27cd-4551-96a0-2a85e92fb5d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:57.344700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271efaa-27cd-4551-96a0-2a85e92fb5d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:18:57.345009 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271efaa-27cd-4551-96a0-2a85e92fb5d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:57.346631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271efaa-27cd-4551-96a0-2a85e92fb5d0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:57.359369 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1195/2358] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:18:57.364163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c630b3c-5a4b-4c87-8939-a08497dc8c8e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:57.378978 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c630b3c-5a4b-4c87-8939-a08497dc8c8e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:57.386029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a550dad-55f7-4fc4-b93e-95773ac7c07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:57.386695 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a550dad-55f7-4fc4-b93e-95773ac7c07c admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:57.386972 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a550dad-55f7-4fc4-b93e-95773ac7c07c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:57.387243 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a550dad-55f7-4fc4-b93e-95773ac7c07c admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:57.387993 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a550dad-55f7-4fc4-b93e-95773ac7c07c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:57.390405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a550dad-55f7-4fc4-b93e-95773ac7c07c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:57.421047 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1164/2359] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:18:57 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:18:57.425318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271efaa-27cd-4551-96a0-2a85e92fb5d0 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:57.435772 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271efaa-27cd-4551-96a0-2a85e92fb5d0 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:57.496175 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c2cca-4bda-4093-b71e-f87946adef46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:57.496763 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c2cca-4bda-4093-b71e-f87946adef46 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:57.497043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c2cca-4bda-4093-b71e-f87946adef46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:57.497313 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c2cca-4bda-4093-b71e-f87946adef46 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:57.498009 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c2cca-4bda-4093-b71e-f87946adef46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:57.499969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c2cca-4bda-4093-b71e-f87946adef46 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:57.526151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1196/2360] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:18:57 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:18:57.532447 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a550dad-55f7-4fc4-b93e-95773ac7c07c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:57.546196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a550dad-55f7-4fc4-b93e-95773ac7c07c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:57.553196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d51887-cf9e-4b34-b379-831b2f0e81df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:57.554161 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d51887-cf9e-4b34-b379-831b2f0e81df admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:57.554440 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d51887-cf9e-4b34-b379-831b2f0e81df admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:57.554708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d51887-cf9e-4b34-b379-831b2f0e81df admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:57.555435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d51887-cf9e-4b34-b379-831b2f0e81df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:57.557084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d51887-cf9e-4b34-b379-831b2f0e81df admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:57.569247 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1165/2361] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:18:57.573940 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c2cca-4bda-4093-b71e-f87946adef46 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:57.585048 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c2cca-4bda-4093-b71e-f87946adef46 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:57.591945 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295f574-ffdb-4e0a-b0b2-59093ffa87e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:57.592711 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295f574-ffdb-4e0a-b0b2-59093ffa87e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:57.593264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295f574-ffdb-4e0a-b0b2-59093ffa87e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:57.593439 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295f574-ffdb-4e0a-b0b2-59093ffa87e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:57.594229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6295f574-ffdb-4e0a-b0b2-59093ffa87e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:57.595932 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6295f574-ffdb-4e0a-b0b2-59093ffa87e3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:57.610417 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1197/2362] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:18:57.614509 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d51887-cf9e-4b34-b379-831b2f0e81df admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:57.625027 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d51887-cf9e-4b34-b379-831b2f0e81df admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:57.629994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca2ded-bb70-41d4-b5f6-08b6414e7b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:57.630906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca2ded-bb70-41d4-b5f6-08b6414e7b59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:57.631186 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca2ded-bb70-41d4-b5f6-08b6414e7b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:57.631460 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca2ded-bb70-41d4-b5f6-08b6414e7b59 admin admin] PATH_INFO: `/v3/projects/7457a8f362994239ad37f279eb57cf0a/users/382e7a3612e348cc96c95dd3be986061/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:57.652237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca2ded-bb70-41d4-b5f6-08b6414e7b59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7457a8f362994239ad37f279eb57cf0a, user_id=382e7a3612e348cc96c95dd3be986061, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:57.654840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca2ded-bb70-41d4-b5f6-08b6414e7b59 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:57.668903 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1166/2363] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:18:57 2022] PUT /identity/v3/projects/7457a8f362994239ad37f279eb57cf0a/users/382e7a3612e348cc96c95dd3be986061/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:57.674021 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2766d-79ab-4154-ad10-ef792992d562 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:57.674311 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2766d-79ab-4154-ad10-ef792992d562 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:57.674646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2766d-79ab-4154-ad10-ef792992d562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:57.691871 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-1ae2766d-79ab-4154-ad10-ef792992d562 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:57.693104 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1ae2766d-79ab-4154-ad10-ef792992d562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:57.764596 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1198/2364] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:18:57.774199 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca2ded-bb70-41d4-b5f6-08b6414e7b59 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:57.775997 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-38ca2ded-bb70-41d4-b5f6-08b6414e7b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:57.797989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:57.836866 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca2ded-bb70-41d4-b5f6-08b6414e7b59 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:57.840826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b92b7-9d09-47d1-adcb-ae138cfac4c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:57.841490 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b92b7-9d09-47d1-adcb-ae138cfac4c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:57.842122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b92b7-9d09-47d1-adcb-ae138cfac4c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:57.842122 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b92b7-9d09-47d1-adcb-ae138cfac4c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:57.852990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b92b7-9d09-47d1-adcb-ae138cfac4c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:57.854473 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b92b7-9d09-47d1-adcb-ae138cfac4c3 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:57.864408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1167/2365] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:58.912738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe041006-44d2-41e7-9ad9-f41f14fb189a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:58.913335 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe041006-44d2-41e7-9ad9-f41f14fb189a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:58.913438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b92b7-9d09-47d1-adcb-ae138cfac4c3 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:58.913801 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe041006-44d2-41e7-9ad9-f41f14fb189a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:58.914843 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b23b92b7-9d09-47d1-adcb-ae138cfac4c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:58.955083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-fe041006-44d2-41e7-9ad9-f41f14fb189a None None] MFA Rules not processed for user `382e7a3612e348cc96c95dd3be986061`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:58.956764 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fe041006-44d2-41e7-9ad9-f41f14fb189a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:58.965903 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:58.981726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b92b7-9d09-47d1-adcb-ae138cfac4c3 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:58.986961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1539122-0fba-498d-b7f7-144a9c3301bc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b43db1fd0b494a3f8588b498b585f247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:58.987664 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1539122-0fba-498d-b7f7-144a9c3301bc service glance] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:58.987952 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1539122-0fba-498d-b7f7-144a9c3301bc service glance] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:58.988225 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1539122-0fba-498d-b7f7-144a9c3301bc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:58.991781 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b1539122-0fba-498d-b7f7-144a9c3301bc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:59.052291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1199/2366] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:18:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:59.064508 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe041006-44d2-41e7-9ad9-f41f14fb189a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:59.066378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fe041006-44d2-41e7-9ad9-f41f14fb189a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:59.109117 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1539122-0fba-498d-b7f7-144a9c3301bc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:59.111299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1539122-0fba-498d-b7f7-144a9c3301bc service glance] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:59.153585 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1168/2367] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:59.155921 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe041006-44d2-41e7-9ad9-f41f14fb189a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:59.160016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38cc1d0-5496-4d80-ba04-47d20efd78a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:59.160623 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38cc1d0-5496-4d80-ba04-47d20efd78a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:59.160906 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38cc1d0-5496-4d80-ba04-47d20efd78a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:59.161180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38cc1d0-5496-4d80-ba04-47d20efd78a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:59.171348 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38cc1d0-5496-4d80-ba04-47d20efd78a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:59.172989 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38cc1d0-5496-4d80-ba04-47d20efd78a7 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:59.196163 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1200/2368] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:18:59.274265 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f79b2-9427-41a3-8468-944d3881306d None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:59.274556 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f79b2-9427-41a3-8468-944d3881306d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:59.276457 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f79b2-9427-41a3-8468-944d3881306d None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:59.277871 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1169/2369] 10.222.0.27 () {54 vars in 921 bytes} [Mon Jun 27 21:18:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:18:59.283904 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6ab21-a4be-458c-9929-b0b0b4b0e4a8 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:59.284236 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6ab21-a4be-458c-9929-b0b0b4b0e4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:59.284605 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6ab21-a4be-458c-9929-b0b0b4b0e4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:59.316854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-14f6ab21-a4be-458c-9929-b0b0b4b0e4a8 None None] MFA Rules not processed for user `9c7ce29db6b14176a4c9be436bcddc6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:59.318733 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-14f6ab21-a4be-458c-9929-b0b0b4b0e4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:59.330876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1201/2370] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Jun 27 21:18:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:59.342566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f79b2-9427-41a3-8468-944d3881306d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:59.354370 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f79b2-9427-41a3-8468-944d3881306d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:59.358925 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4032dd-7af7-4f8c-8a26-9146a3a31a5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:59.359990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4032dd-7af7-4f8c-8a26-9146a3a31a5f service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:59.360409 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4032dd-7af7-4f8c-8a26-9146a3a31a5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:59.360684 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4032dd-7af7-4f8c-8a26-9146a3a31a5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:59.369881 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4032dd-7af7-4f8c-8a26-9146a3a31a5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:59.371665 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4032dd-7af7-4f8c-8a26-9146a3a31a5f service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:59.380830 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1170/2371] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:59.472716 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab7976-7297-4386-9c2d-6e2073da64f5 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:59.472716 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab7976-7297-4386-9c2d-6e2073da64f5 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:59.472716 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab7976-7297-4386-9c2d-6e2073da64f5 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:59.473014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1202/2372] 10.222.0.27 () {54 vars in 921 bytes} [Mon Jun 27 21:18:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:18:59.479288 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f753a3-82c0-460c-98c4-629dbb220494 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:59.479579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f753a3-82c0-460c-98c4-629dbb220494 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:59.479883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f753a3-82c0-460c-98c4-629dbb220494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:59.527970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-12f753a3-82c0-460c-98c4-629dbb220494 None None] MFA Rules not processed for user `8d7fe01ef64f46f6a0e77d5945219b33`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:59.529575 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-12f753a3-82c0-460c-98c4-629dbb220494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:59.541696 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1171/2373] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Jun 27 21:18:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:59.554391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab7976-7297-4386-9c2d-6e2073da64f5 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:59.567095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab7976-7297-4386-9c2d-6e2073da64f5 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:59.571234 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf8b748-bf1d-416b-8e10-b69b138b9c23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:59.572139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8b748-bf1d-416b-8e10-b69b138b9c23 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:59.572424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8b748-bf1d-416b-8e10-b69b138b9c23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:59.572693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8b748-bf1d-416b-8e10-b69b138b9c23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:59.583147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf8b748-bf1d-416b-8e10-b69b138b9c23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:59.584618 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf8b748-bf1d-416b-8e10-b69b138b9c23 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:59.596159 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1203/2374] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:19:00.584919 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edd33d-7ca8-4f9e-9db2-dc422c7ea127 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:19:00.585493 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edd33d-7ca8-4f9e-9db2-dc422c7ea127 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:19:00.585493 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edd33d-7ca8-4f9e-9db2-dc422c7ea127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:19:00.592831 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:19:00.604945 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-06edd33d-7ca8-4f9e-9db2-dc422c7ea127 None None] MFA Rules not processed for user `382e7a3612e348cc96c95dd3be986061`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:19:00.606162 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-06edd33d-7ca8-4f9e-9db2-dc422c7ea127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:19:00.613068 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1172/2375] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:19:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:19:05.703376 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54718e5-b808-42d3-9b8b-89499501a067 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:19:05.703945 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54718e5-b808-42d3-9b8b-89499501a067 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:19:05.704041 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54718e5-b808-42d3-9b8b-89499501a067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:19:05.721732 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b54718e5-b808-42d3-9b8b-89499501a067 None None] MFA Rules not processed for user `32b9a1c24fe84019a311b1331ea2c004`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:19:05.723083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b54718e5-b808-42d3-9b8b-89499501a067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:19:05.784896 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1204/2376] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:19:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:19:05.794185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b86f7-9e4c-4ab7-a178-72041332916a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:19:05.795447 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b86f7-9e4c-4ab7-a178-72041332916a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:19:05.795751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b86f7-9e4c-4ab7-a178-72041332916a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:19:05.817245 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-738b86f7-9e4c-4ab7-a178-72041332916a None None] MFA Rules not processed for user `e802da0b078e457da466a9a7b0f15085`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:19:05.818492 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-738b86f7-9e4c-4ab7-a178-72041332916a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:19:05.885827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1173/2377] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Jun 27 21:19:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:19:05.914011 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54718e5-b808-42d3-9b8b-89499501a067 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:19:05.915960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b54718e5-b808-42d3-9b8b-89499501a067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:19:05.950868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:19:06.016939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54718e5-b808-42d3-9b8b-89499501a067 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:19:06.027173 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5573e-1b0a-4ad6-9f21-2feaff83293a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28acb0322114aebafcf62bdf9a346df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:19:06.028193 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5573e-1b0a-4ad6-9f21-2feaff83293a service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:19:06.028625 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5573e-1b0a-4ad6-9f21-2feaff83293a service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:19:06.029255 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5573e-1b0a-4ad6-9f21-2feaff83293a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:19:06.044487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5573e-1b0a-4ad6-9f21-2feaff83293a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:19:06.047171 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5573e-1b0a-4ad6-9f21-2feaff83293a service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:19:06.057255 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1205/2378] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Jun 27 21:19:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:19:15.873983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=26 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:19:16.053214 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=19 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:19:20.525764 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b86f7-9e4c-4ab7-a178-72041332916a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:19:20.526274 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5573e-1b0a-4ad6-9f21-2feaff83293a service swift] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:19:20.527419 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-dcd5573e-1b0a-4ad6-9f21-2feaff83293a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:19:20.527575 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-738b86f7-9e4c-4ab7-a178-72041332916a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:19:20.631295 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5573e-1b0a-4ad6-9f21-2feaff83293a service swift] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:19:20.637271 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fded4e-bf6c-4726-b79a-cde73c642cad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5fa9b0b0314ee9bb7e16b8464307b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:19:20.638242 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fded4e-bf6c-4726-b79a-cde73c642cad service placement] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:19:20.638801 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fded4e-bf6c-4726-b79a-cde73c642cad service placement] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:19:20.639106 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fded4e-bf6c-4726-b79a-cde73c642cad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:19:20.641119 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-73fded4e-bf6c-4726-b79a-cde73c642cad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:19:20.641688 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b86f7-9e4c-4ab7-a178-72041332916a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:19:20.647484 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ee895-21fd-4fca-9a7c-0e6d94c88ccd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5fa9b0b0314ee9bb7e16b8464307b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:19:20.648245 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ee895-21fd-4fca-9a7c-0e6d94c88ccd service placement] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:19:20.648617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ee895-21fd-4fca-9a7c-0e6d94c88ccd service placement] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:19:20.648964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ee895-21fd-4fca-9a7c-0e6d94c88ccd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:19:20.653204 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fded4e-bf6c-4726-b79a-cde73c642cad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:19:20.655033 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fded4e-bf6c-4726-b79a-cde73c642cad service placement] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:19:20.658243 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ee895-21fd-4fca-9a7c-0e6d94c88ccd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:19:20.659868 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ee895-21fd-4fca-9a7c-0e6d94c88ccd service placement] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:19:20.667083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1206/2379] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:19:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:19:20.670289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1174/2380] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:19:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:19:24.075317 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be924c3-0ba9-46af-832c-5b6b26b6b417 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:19:24.076019 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be924c3-0ba9-46af-832c-5b6b26b6b417 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:19:24.076019 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be924c3-0ba9-46af-832c-5b6b26b6b417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:19:24.103256 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8be924c3-0ba9-46af-832c-5b6b26b6b417 None None] MFA Rules not processed for user `92c1e4ea6ab84f94a2215f8c534ca159`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:19:24.105026 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8be924c3-0ba9-46af-832c-5b6b26b6b417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:19:24.194559 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1207/2381] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:19:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:19:30.670627 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=41 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:19:34.180798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=66 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:19:35.399305 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7555d78-7a1a-424e-998a-95fa501578cd None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:19:35.399826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7555d78-7a1a-424e-998a-95fa501578cd None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:19:35.399926 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7555d78-7a1a-424e-998a-95fa501578cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:19:35.465231 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-f7555d78-7a1a-424e-998a-95fa501578cd None None] MFA Rules not processed for user `e802da0b078e457da466a9a7b0f15085`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:19:35.467322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f7555d78-7a1a-424e-998a-95fa501578cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:19:35.490377 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1175/2382] 10.222.0.27 () {58 vars in 967 bytes} [Mon Jun 27 21:19:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:19:35.518691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be924c3-0ba9-46af-832c-5b6b26b6b417 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:19:35.557981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be924c3-0ba9-46af-832c-5b6b26b6b417 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:19:35.563520 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cbe72-b348-47c8-bc31-63a99551c08a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28acb0322114aebafcf62bdf9a346df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:19:35.564157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cbe72-b348-47c8-bc31-63a99551c08a service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:19:35.564438 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cbe72-b348-47c8-bc31-63a99551c08a service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:19:35.564709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cbe72-b348-47c8-bc31-63a99551c08a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:19:35.575128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76cbe72-b348-47c8-bc31-63a99551c08a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:19:35.576784 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76cbe72-b348-47c8-bc31-63a99551c08a service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:19:35.588770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1208/2383] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Jun 27 21:19:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:19:41.281089 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0183bd-0461-4a6b-a67b-00c15632f734 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:19:41.281581 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0183bd-0461-4a6b-a67b-00c15632f734 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:19:41.281753 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0183bd-0461-4a6b-a67b-00c15632f734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:19:41.299820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-cf0183bd-0461-4a6b-a67b-00c15632f734 None None] MFA Rules not processed for user `30cbde26a1964f22902c24917006fb90`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:19:41.300996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-cf0183bd-0461-4a6b-a67b-00c15632f734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:19:41.396115 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1176/2384] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:19:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:19:41.840653 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cbe72-b348-47c8-bc31-63a99551c08a service swift] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:19:41.850858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cbe72-b348-47c8-bc31-63a99551c08a service swift] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:19:41.855031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc02e88-4251-4e7f-adce-7fec6783261f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:19:41.856274 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc02e88-4251-4e7f-adce-7fec6783261f service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:19:41.856274 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc02e88-4251-4e7f-adce-7fec6783261f service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:19:41.856595 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc02e88-4251-4e7f-adce-7fec6783261f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:19:41.858872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bbc02e88-4251-4e7f-adce-7fec6783261f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:19:41.930477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc02e88-4251-4e7f-adce-7fec6783261f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:19:41.932097 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc02e88-4251-4e7f-adce-7fec6783261f service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:19:41.942224 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1209/2385] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:19:41 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:19:43.949447 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0183bd-0461-4a6b-a67b-00c15632f734 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:19:43.959638 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0183bd-0461-4a6b-a67b-00c15632f734 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:19:43.963128 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495ce8b-b0ff-4479-a625-29992591ae21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:19:43.963811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495ce8b-b0ff-4479-a625-29992591ae21 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:19:43.964089 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495ce8b-b0ff-4479-a625-29992591ae21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:19:43.964362 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495ce8b-b0ff-4479-a625-29992591ae21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:19:43.966266 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4495ce8b-b0ff-4479-a625-29992591ae21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:19:44.039232 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495ce8b-b0ff-4479-a625-29992591ae21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:19:44.040771 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495ce8b-b0ff-4479-a625-29992591ae21 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:19:44.050765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1177/2386] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:19:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:19:51.937790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=40 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:19:54.046798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=73 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:20:28.116025 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc2f34-3579-467e-8641-26d7bb15abdb None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:28.116651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc2f34-3579-467e-8641-26d7bb15abdb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:28.116651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc2f34-3579-467e-8641-26d7bb15abdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:28.144522 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-69bc2f34-3579-467e-8641-26d7bb15abdb None None] MFA Rules not processed for user `382e7a3612e348cc96c95dd3be986061`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:20:28.145733 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-69bc2f34-3579-467e-8641-26d7bb15abdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:20:28.153801 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1210/2387] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:20:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:20:38.136735 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:20:49.458336 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f5807-29b1-4295-b755-440a196d33bb None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:49.458853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f5807-29b1-4295-b755-440a196d33bb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:49.458950 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f5807-29b1-4295-b755-440a196d33bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:49.481104 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-8d3f5807-29b1-4295-b755-440a196d33bb None None] MFA Rules not processed for user `74b2295521bc45509ef6b68c1885ac6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:20:49.482336 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8d3f5807-29b1-4295-b755-440a196d33bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:20:49.542992 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1178/2388] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:20:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:20:53.504050 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc2f34-3579-467e-8641-26d7bb15abdb None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:20:53.505472 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-69bc2f34-3579-467e-8641-26d7bb15abdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:20:53.518806 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc2f34-3579-467e-8641-26d7bb15abdb None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:20:53.522463 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f87697-41c4-4e27-be31-2fb17cceff51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:20:53.523223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f87697-41c4-4e27-be31-2fb17cceff51 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:53.523563 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f87697-41c4-4e27-be31-2fb17cceff51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:53.523927 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f87697-41c4-4e27-be31-2fb17cceff51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:53.525739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-33f87697-41c4-4e27-be31-2fb17cceff51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:20:53.540155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f87697-41c4-4e27-be31-2fb17cceff51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:20:53.541596 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f87697-41c4-4e27-be31-2fb17cceff51 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:20:53.552769 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1211/2389] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:20:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:20:54.793945 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f5807-29b1-4295-b755-440a196d33bb None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:20:54.805164 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f5807-29b1-4295-b755-440a196d33bb None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:20:54.809430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f3112-92b8-43f9-a222-5a012970f3b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:20:54.810070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f3112-92b8-43f9-a222-5a012970f3b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:54.810352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f3112-92b8-43f9-a222-5a012970f3b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:54.810620 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f3112-92b8-43f9-a222-5a012970f3b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:54.812863 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-356f3112-92b8-43f9-a222-5a012970f3b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:20:54.827407 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f3112-92b8-43f9-a222-5a012970f3b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:20:54.828876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f3112-92b8-43f9-a222-5a012970f3b4 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:20:54.839926 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1179/2390] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:20:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:03.548312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=6 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:04.835213 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=31 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:05.640990 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f87697-41c4-4e27-be31-2fb17cceff51 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:05.642458 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-33f87697-41c4-4e27-be31-2fb17cceff51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:05.661220 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f87697-41c4-4e27-be31-2fb17cceff51 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:05.667166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeade02f-a7d5-40c6-a0a4-d2291cd35e80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:05.667918 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeade02f-a7d5-40c6-a0a4-d2291cd35e80 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:05.668199 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeade02f-a7d5-40c6-a0a4-d2291cd35e80 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:05.668456 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeade02f-a7d5-40c6-a0a4-d2291cd35e80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:05.671073 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-aeade02f-a7d5-40c6-a0a4-d2291cd35e80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:05.684145 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeade02f-a7d5-40c6-a0a4-d2291cd35e80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:05.685815 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeade02f-a7d5-40c6-a0a4-d2291cd35e80 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:05.695579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1212/2391] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:07.060014 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f3112-92b8-43f9-a222-5a012970f3b4 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:07.080859 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f3112-92b8-43f9-a222-5a012970f3b4 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:07.084776 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98ea41-73ac-4f83-b046-9f6d271b9444 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:07.085431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98ea41-73ac-4f83-b046-9f6d271b9444 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:07.085715 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98ea41-73ac-4f83-b046-9f6d271b9444 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:07.086155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98ea41-73ac-4f83-b046-9f6d271b9444 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:07.087984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1d98ea41-73ac-4f83-b046-9f6d271b9444 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:07.100647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98ea41-73ac-4f83-b046-9f6d271b9444 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:07.102282 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98ea41-73ac-4f83-b046-9f6d271b9444 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:07.112618 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1180/2392] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:15.691203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=6 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:17.107902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=10 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:23.915769 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeade02f-a7d5-40c6-a0a4-d2291cd35e80 service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:23.927651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeade02f-a7d5-40c6-a0a4-d2291cd35e80 service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:23.931922 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2da6bd-94fc-4d9a-a55d-d691bc4b4365 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:23.932601 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2da6bd-94fc-4d9a-a55d-d691bc4b4365 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:23.932886 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2da6bd-94fc-4d9a-a55d-d691bc4b4365 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:23.933202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2da6bd-94fc-4d9a-a55d-d691bc4b4365 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:23.935179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-be2da6bd-94fc-4d9a-a55d-d691bc4b4365 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:23.947920 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2da6bd-94fc-4d9a-a55d-d691bc4b4365 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:23.949540 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2da6bd-94fc-4d9a-a55d-d691bc4b4365 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:23.958530 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1213/2393] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:21:24.938812 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98ea41-73ac-4f83-b046-9f6d271b9444 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:24.951514 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98ea41-73ac-4f83-b046-9f6d271b9444 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:24.957121 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470272ad-8112-44de-8441-8c1888185765 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:24.957788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470272ad-8112-44de-8441-8c1888185765 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:24.958073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470272ad-8112-44de-8441-8c1888185765 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:24.958349 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470272ad-8112-44de-8441-8c1888185765 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:24.960553 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-470272ad-8112-44de-8441-8c1888185765 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:24.973144 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470272ad-8112-44de-8441-8c1888185765 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:24.974762 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470272ad-8112-44de-8441-8c1888185765 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:24.986783 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1181/2394] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:26.499550 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38589d2b-194b-47f5-8f42-1dd132487a12 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:26.500165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38589d2b-194b-47f5-8f42-1dd132487a12 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:26.500165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38589d2b-194b-47f5-8f42-1dd132487a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:26.519337 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-38589d2b-194b-47f5-8f42-1dd132487a12 None None] MFA Rules not processed for user `92c1e4ea6ab84f94a2215f8c534ca159`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:26.520592 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-38589d2b-194b-47f5-8f42-1dd132487a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:26.529484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1214/2395] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:21:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:28.653786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f2cdd-fe9a-42cb-8b69-0c3df274e9ca None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:28.654330 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f2cdd-fe9a-42cb-8b69-0c3df274e9ca None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:28.654330 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f2cdd-fe9a-42cb-8b69-0c3df274e9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:28.673587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a54f2cdd-fe9a-42cb-8b69-0c3df274e9ca None None] MFA Rules not processed for user `30cbde26a1964f22902c24917006fb90`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:28.674791 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a54f2cdd-fe9a-42cb-8b69-0c3df274e9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:28.682488 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1182/2396] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:21:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:28.998305 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38589d2b-194b-47f5-8f42-1dd132487a12 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:29.014196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38589d2b-194b-47f5-8f42-1dd132487a12 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:29.019081 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a2bc3-48b2-48f2-9af2-daf2be6b4a54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:29.019801 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a2bc3-48b2-48f2-9af2-daf2be6b4a54 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:29.020087 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a2bc3-48b2-48f2-9af2-daf2be6b4a54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:29.020354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a2bc3-48b2-48f2-9af2-daf2be6b4a54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:29.022476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ad5a2bc3-48b2-48f2-9af2-daf2be6b4a54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:29.032296 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a2bc3-48b2-48f2-9af2-daf2be6b4a54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:29.033741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a2bc3-48b2-48f2-9af2-daf2be6b4a54 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:29.042440 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1215/2397] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:29.396285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f2cdd-fe9a-42cb-8b69-0c3df274e9ca None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:29.410969 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f2cdd-fe9a-42cb-8b69-0c3df274e9ca None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:29.417045 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d8fe9-71de-421e-96d5-d41b17c75813 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5fa9b0b0314ee9bb7e16b8464307b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:29.417913 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d8fe9-71de-421e-96d5-d41b17c75813 service placement] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:29.418312 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d8fe9-71de-421e-96d5-d41b17c75813 service placement] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:29.418685 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d8fe9-71de-421e-96d5-d41b17c75813 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:29.420874 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-337d8fe9-71de-421e-96d5-d41b17c75813 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:29.434898 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d8fe9-71de-421e-96d5-d41b17c75813 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:29.437132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d8fe9-71de-421e-96d5-d41b17c75813 service placement] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:29.448172 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1183/2398] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:32.283385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a2bc3-48b2-48f2-9af2-daf2be6b4a54 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:32.296697 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a2bc3-48b2-48f2-9af2-daf2be6b4a54 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:32.301569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b39e91-9475-4144-b16c-a0ef2b5c3628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:32.302504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b39e91-9475-4144-b16c-a0ef2b5c3628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:32.302791 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b39e91-9475-4144-b16c-a0ef2b5c3628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:32.303071 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b39e91-9475-4144-b16c-a0ef2b5c3628 admin admin] PATH_INFO: `/v3/users/382e7a3612e348cc96c95dd3be986061` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:32.305747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b39e91-9475-4144-b16c-a0ef2b5c3628 admin admin] RBAC: Authorizing `identity:delete_user(user_id=382e7a3612e348cc96c95dd3be986061)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:32.307740 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b39e91-9475-4144-b16c-a0ef2b5c3628 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:32.378728 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49b39e91-9475-4144-b16c-a0ef2b5c3628 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '382e7a3612e348cc96c95dd3be986061'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:32.383067 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49b39e91-9475-4144-b16c-a0ef2b5c3628 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '382e7a3612e348cc96c95dd3be986061'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:32.389801 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49b39e91-9475-4144-b16c-a0ef2b5c3628 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '382e7a3612e348cc96c95dd3be986061'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:32.416019 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49b39e91-9475-4144-b16c-a0ef2b5c3628 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '382e7a3612e348cc96c95dd3be986061'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:32.426221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49b39e91-9475-4144-b16c-a0ef2b5c3628 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '382e7a3612e348cc96c95dd3be986061'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:32.442411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1216/2399] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:21:32 2022] DELETE /identity/v3/users/382e7a3612e348cc96c95dd3be986061 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:32.625341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d8fe9-71de-421e-96d5-d41b17c75813 service placement] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:32.643273 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d8fe9-71de-421e-96d5-d41b17c75813 service placement] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:32.704130 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4bf5f-0e3c-4349-bacc-60869dd49f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:32.705008 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4bf5f-0e3c-4349-bacc-60869dd49f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:32.705292 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4bf5f-0e3c-4349-bacc-60869dd49f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:32.705571 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4bf5f-0e3c-4349-bacc-60869dd49f43 admin admin] PATH_INFO: `/v3/projects/7457a8f362994239ad37f279eb57cf0a` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:32.706761 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4bf5f-0e3c-4349-bacc-60869dd49f43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7457a8f362994239ad37f279eb57cf0a)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:32.708828 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4bf5f-0e3c-4349-bacc-60869dd49f43 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:32.757224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-0da4bf5f-0e3c-4349-bacc-60869dd49f43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7457a8f362994239ad37f279eb57cf0a', 'request_id': 'req-0da4bf5f-0e3c-4349-bacc-60869dd49f43'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:32.766139 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-0da4bf5f-0e3c-4349-bacc-60869dd49f43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7457a8f362994239ad37f279eb57cf0a', 'request_id': 'req-0da4bf5f-0e3c-4349-bacc-60869dd49f43'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:32.772717 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-0da4bf5f-0e3c-4349-bacc-60869dd49f43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7457a8f362994239ad37f279eb57cf0a', 'request_id': 'req-0da4bf5f-0e3c-4349-bacc-60869dd49f43'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:32.784966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-0da4bf5f-0e3c-4349-bacc-60869dd49f43 admin admin] The token cache is being invalidate because project 7457a8f362994239ad37f279eb57cf0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:21:32.785190 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-0da4bf5f-0e3c-4349-bacc-60869dd49f43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:32.785983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-0da4bf5f-0e3c-4349-bacc-60869dd49f43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:32.787551 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1184/2400] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:21:32 2022] DELETE /identity/v3/projects/7457a8f362994239ad37f279eb57cf0a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:42.438043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=44,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:42.441098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b39e91-9475-4144-b16c-a0ef2b5c3628 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:42.443221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-49b39e91-9475-4144-b16c-a0ef2b5c3628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:42.521158 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b39e91-9475-4144-b16c-a0ef2b5c3628 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:42.583847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7937e-99d4-4475-984f-4c82f4b293df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:42.584416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7937e-99d4-4475-984f-4c82f4b293df service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:42.584696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7937e-99d4-4475-984f-4c82f4b293df service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:42.584966 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7937e-99d4-4475-984f-4c82f4b293df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:42.586787 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cea7937e-99d4-4475-984f-4c82f4b293df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:42.600938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7937e-99d4-4475-984f-4c82f4b293df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:42.602535 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7937e-99d4-4475-984f-4c82f4b293df service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:42.613963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1217/2401] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:42.781529 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=66,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:52.610138 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=62 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:10.676099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4bf5f-0e3c-4349-bacc-60869dd49f43 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:10.677636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0da4bf5f-0e3c-4349-bacc-60869dd49f43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:10.695905 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4bf5f-0e3c-4349-bacc-60869dd49f43 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:10.701650 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6b67d-ab48-4dbb-a279-555c8b418a16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:10.702396 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6b67d-ab48-4dbb-a279-555c8b418a16 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:10.702678 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6b67d-ab48-4dbb-a279-555c8b418a16 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:10.702952 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6b67d-ab48-4dbb-a279-555c8b418a16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:10.704968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f2c6b67d-ab48-4dbb-a279-555c8b418a16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:10.718950 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c6b67d-ab48-4dbb-a279-555c8b418a16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:10.720890 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c6b67d-ab48-4dbb-a279-555c8b418a16 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:10.733540 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1185/2402] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:11.250028 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327331b-cbb5-432a-94b5-c121ce498568 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:11.250595 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327331b-cbb5-432a-94b5-c121ce498568 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:11.250857 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327331b-cbb5-432a-94b5-c121ce498568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:11.270608 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b327331b-cbb5-432a-94b5-c121ce498568 None None] MFA Rules not processed for user `e802da0b078e457da466a9a7b0f15085`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:11.272491 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b327331b-cbb5-432a-94b5-c121ce498568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:11.343086 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1218/2403] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Jun 27 21:22:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:11.400478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6b67d-ab48-4dbb-a279-555c8b418a16 service nova] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:11.401973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f2c6b67d-ab48-4dbb-a279-555c8b418a16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:11.502442 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6b67d-ab48-4dbb-a279-555c8b418a16 service nova] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:11.508755 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840afa5a-b9c8-4d4e-8b45-593f058e27a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28acb0322114aebafcf62bdf9a346df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:11.509739 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840afa5a-b9c8-4d4e-8b45-593f058e27a4 service swift] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:11.510152 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840afa5a-b9c8-4d4e-8b45-593f058e27a4 service swift] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:11.510545 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840afa5a-b9c8-4d4e-8b45-593f058e27a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:11.523062 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840afa5a-b9c8-4d4e-8b45-593f058e27a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:11.524862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840afa5a-b9c8-4d4e-8b45-593f058e27a4 service swift] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:11.534590 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1186/2404] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Jun 27 21:22:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:22:18.398397 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e996b5d-ec25-42f6-b30b-581de3f1308a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:18.398958 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e996b5d-ec25-42f6-b30b-581de3f1308a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:18.398958 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e996b5d-ec25-42f6-b30b-581de3f1308a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:18.418328 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6e996b5d-ec25-42f6-b30b-581de3f1308a None None] MFA Rules not processed for user `32b9a1c24fe84019a311b1331ea2c004`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:18.419705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6e996b5d-ec25-42f6-b30b-581de3f1308a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:18.522850 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1219/2405] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:22:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:21.530952 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=43 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:28.495621 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275b579-598d-4cf3-bfe2-4b023f9b20fe None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:28.496296 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275b579-598d-4cf3-bfe2-4b023f9b20fe None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:28.496296 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275b579-598d-4cf3-bfe2-4b023f9b20fe None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:28.497795 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1187/2406] 10.222.0.27 () {54 vars in 921 bytes} [Mon Jun 27 21:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:22:28.499226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=50 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:28.504147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072429f2-639f-43a4-b9d8-1ade27dece7f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:28.504627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072429f2-639f-43a4-b9d8-1ade27dece7f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:28.504912 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072429f2-639f-43a4-b9d8-1ade27dece7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:28.532919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-072429f2-639f-43a4-b9d8-1ade27dece7f None None] MFA Rules not processed for user `8d7fe01ef64f46f6a0e77d5945219b33`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:28.534165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-072429f2-639f-43a4-b9d8-1ade27dece7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:28.543599 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1220/2407] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Jun 27 21:22:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:28.554948 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275b579-598d-4cf3-bfe2-4b023f9b20fe None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:28.556838 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1275b579-598d-4cf3-bfe2-4b023f9b20fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:28.642781 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275b579-598d-4cf3-bfe2-4b023f9b20fe None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:28.647292 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb48620-27cd-4712-a8c0-17ee84d65eba service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b43db1fd0b494a3f8588b498b585f247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:28.647996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb48620-27cd-4712-a8c0-17ee84d65eba service glance] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:28.648281 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb48620-27cd-4712-a8c0-17ee84d65eba service glance] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:28.648555 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb48620-27cd-4712-a8c0-17ee84d65eba service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:28.659548 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb48620-27cd-4712-a8c0-17ee84d65eba service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:28.661017 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb48620-27cd-4712-a8c0-17ee84d65eba service glance] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:28.674089 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1188/2408] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:28.756564 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022af9a-d4aa-4fd0-ba56-22baa636defc None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:28.756852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022af9a-d4aa-4fd0-ba56-22baa636defc None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:28.757131 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022af9a-d4aa-4fd0-ba56-22baa636defc None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:28.758526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1221/2409] 10.222.0.27 () {54 vars in 921 bytes} [Mon Jun 27 21:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:22:28.763425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb147346-c8eb-4998-857f-aef4edfbeabb None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:28.763820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb147346-c8eb-4998-857f-aef4edfbeabb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:28.764099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb147346-c8eb-4998-857f-aef4edfbeabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:28.785174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-fb147346-c8eb-4998-857f-aef4edfbeabb None None] MFA Rules not processed for user `9c7ce29db6b14176a4c9be436bcddc6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:28.786575 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fb147346-c8eb-4998-857f-aef4edfbeabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:28.845744 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1189/2410] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Jun 27 21:22:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:28.857707 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022af9a-d4aa-4fd0-ba56-22baa636defc None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:28.859536 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4022af9a-d4aa-4fd0-ba56-22baa636defc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:28.873868 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022af9a-d4aa-4fd0-ba56-22baa636defc None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:28.877875 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91430c-b327-4737-8dc3-6a9969b752aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:28.878515 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91430c-b327-4737-8dc3-6a9969b752aa service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:28.878794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91430c-b327-4737-8dc3-6a9969b752aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:28.879072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91430c-b327-4737-8dc3-6a9969b752aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:28.886915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91430c-b327-4737-8dc3-6a9969b752aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:28.888412 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91430c-b327-4737-8dc3-6a9969b752aa service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:28.896734 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1222/2411] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:28.985062 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7682aa-130e-4608-a533-7125e7691c34 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:28.985348 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7682aa-130e-4608-a533-7125e7691c34 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:28.985628 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7682aa-130e-4608-a533-7125e7691c34 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:28.987044 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1190/2412] 10.222.0.27 () {54 vars in 921 bytes} [Mon Jun 27 21:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:22:28.993015 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cf910-63af-4437-a74e-4049b3794555 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:28.993397 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cf910-63af-4437-a74e-4049b3794555 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:28.993737 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cf910-63af-4437-a74e-4049b3794555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:29.013613 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-173cf910-63af-4437-a74e-4049b3794555 None None] MFA Rules not processed for user `8d7fe01ef64f46f6a0e77d5945219b33`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:29.014819 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-173cf910-63af-4437-a74e-4049b3794555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:29.024887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1223/2413] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Jun 27 21:22:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:29.038786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7682aa-130e-4608-a533-7125e7691c34 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:29.041701 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7e7682aa-130e-4608-a533-7125e7691c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:29.068487 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7682aa-130e-4608-a533-7125e7691c34 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:29.075575 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1ead5-632a-48aa-926e-5aa5cdb89f89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:29.076597 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1ead5-632a-48aa-926e-5aa5cdb89f89 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:29.077038 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1ead5-632a-48aa-926e-5aa5cdb89f89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:29.077486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1ead5-632a-48aa-926e-5aa5cdb89f89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:29.095088 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1ead5-632a-48aa-926e-5aa5cdb89f89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:29.096561 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1ead5-632a-48aa-926e-5aa5cdb89f89 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:29.108334 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1191/2414] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:29.215197 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173cf910-63af-4437-a74e-4049b3794555 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:29.216741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-173cf910-63af-4437-a74e-4049b3794555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:29.231462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173cf910-63af-4437-a74e-4049b3794555 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:29.238037 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05699c13-1a61-4b72-9005-68649a764abe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:29.238303 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05699c13-1a61-4b72-9005-68649a764abe service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:29.238627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05699c13-1a61-4b72-9005-68649a764abe service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:29.238968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05699c13-1a61-4b72-9005-68649a764abe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:29.240961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-05699c13-1a61-4b72-9005-68649a764abe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:29.330371 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05699c13-1a61-4b72-9005-68649a764abe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:29.336723 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05699c13-1a61-4b72-9005-68649a764abe service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:29.350664 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1224/2415] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:36.022354 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eec765-64e2-49ec-86cd-6167f6510bb2 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:36.022855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eec765-64e2-49ec-86cd-6167f6510bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:36.022855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eec765-64e2-49ec-86cd-6167f6510bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:36.041961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-98eec765-64e2-49ec-86cd-6167f6510bb2 None None] MFA Rules not processed for user `e802da0b078e457da466a9a7b0f15085`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:36.043195 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-98eec765-64e2-49ec-86cd-6167f6510bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:36.051044 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1192/2416] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Jun 27 21:22:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:36.063584 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05699c13-1a61-4b72-9005-68649a764abe service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:36.074974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05699c13-1a61-4b72-9005-68649a764abe service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:36.079908 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665e6d2-5f23-4bb8-8496-85a3f9cd70fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28acb0322114aebafcf62bdf9a346df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:36.080572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665e6d2-5f23-4bb8-8496-85a3f9cd70fb service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:36.080851 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665e6d2-5f23-4bb8-8496-85a3f9cd70fb service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:36.081127 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665e6d2-5f23-4bb8-8496-85a3f9cd70fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:36.088942 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665e6d2-5f23-4bb8-8496-85a3f9cd70fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:36.090531 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665e6d2-5f23-4bb8-8496-85a3f9cd70fb service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:36.098591 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1225/2417] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Jun 27 21:22:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:22:46.036426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=67 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:46.095509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=46 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:48.840163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eec765-64e2-49ec-86cd-6167f6510bb2 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:48.852115 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eec765-64e2-49ec-86cd-6167f6510bb2 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:48.856870 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f15e5d-188e-4138-82f3-ffa4a8ffbf04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:48.857607 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f15e5d-188e-4138-82f3-ffa4a8ffbf04 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:48.858525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f15e5d-188e-4138-82f3-ffa4a8ffbf04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:48.858993 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f15e5d-188e-4138-82f3-ffa4a8ffbf04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:48.861007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-70f15e5d-188e-4138-82f3-ffa4a8ffbf04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:48.947015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f15e5d-188e-4138-82f3-ffa4a8ffbf04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:48.948732 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f15e5d-188e-4138-82f3-ffa4a8ffbf04 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:48.959166 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1193/2418] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:52.727867 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665e6d2-5f23-4bb8-8496-85a3f9cd70fb service swift] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:52.729405 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1665e6d2-5f23-4bb8-8496-85a3f9cd70fb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:52.811454 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665e6d2-5f23-4bb8-8496-85a3f9cd70fb service swift] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:52.818624 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f15e5d-188e-4138-82f3-ffa4a8ffbf04 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:52.820359 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-70f15e5d-188e-4138-82f3-ffa4a8ffbf04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:52.821588 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74168bbf-1756-4e86-b749-b763c1b949f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5fa9b0b0314ee9bb7e16b8464307b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:52.822497 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74168bbf-1756-4e86-b749-b763c1b949f8 service placement] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:52.822930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74168bbf-1756-4e86-b749-b763c1b949f8 service placement] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:52.823487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74168bbf-1756-4e86-b749-b763c1b949f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:52.825571 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-74168bbf-1756-4e86-b749-b763c1b949f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:52.840761 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f15e5d-188e-4138-82f3-ffa4a8ffbf04 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:52.841303 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74168bbf-1756-4e86-b749-b763c1b949f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:52.843162 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74168bbf-1756-4e86-b749-b763c1b949f8 service placement] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:52.862256 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e390fc-6100-4b5e-9558-9a3c03f150e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:52.863830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1226/2419] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:52.864109 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e390fc-6100-4b5e-9558-9a3c03f150e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:52.864559 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e390fc-6100-4b5e-9558-9a3c03f150e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:52.864991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e390fc-6100-4b5e-9558-9a3c03f150e9 admin admin] PATH_INFO: `/v3/users/92c1e4ea6ab84f94a2215f8c534ca159` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:52.866476 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e390fc-6100-4b5e-9558-9a3c03f150e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92c1e4ea6ab84f94a2215f8c534ca159)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:52.869131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e390fc-6100-4b5e-9558-9a3c03f150e9 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:52.951259 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-11e390fc-6100-4b5e-9558-9a3c03f150e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92c1e4ea6ab84f94a2215f8c534ca159'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:52.952304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-11e390fc-6100-4b5e-9558-9a3c03f150e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92c1e4ea6ab84f94a2215f8c534ca159'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:52.953479 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-11e390fc-6100-4b5e-9558-9a3c03f150e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92c1e4ea6ab84f94a2215f8c534ca159'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:52.969768 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-11e390fc-6100-4b5e-9558-9a3c03f150e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92c1e4ea6ab84f94a2215f8c534ca159'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:52.982966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-11e390fc-6100-4b5e-9558-9a3c03f150e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92c1e4ea6ab84f94a2215f8c534ca159'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:53.028583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1194/2420] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:52 2022] DELETE /identity/v3/users/92c1e4ea6ab84f94a2215f8c534ca159 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:53.411353 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74168bbf-1756-4e86-b749-b763c1b949f8 service placement] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:53.437306 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74168bbf-1756-4e86-b749-b763c1b949f8 service placement] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:53.523532 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306937cc-e1c3-477f-9731-590ddbf7fd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:53.524456 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306937cc-e1c3-477f-9731-590ddbf7fd5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:53.524736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306937cc-e1c3-477f-9731-590ddbf7fd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:53.525012 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306937cc-e1c3-477f-9731-590ddbf7fd5f admin admin] PATH_INFO: `/v3/projects/9c06e827e8c44f58a2051705a46e361c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:53.526294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306937cc-e1c3-477f-9731-590ddbf7fd5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c06e827e8c44f58a2051705a46e361c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:53.528290 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306937cc-e1c3-477f-9731-590ddbf7fd5f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:53.592600 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-306937cc-e1c3-477f-9731-590ddbf7fd5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c06e827e8c44f58a2051705a46e361c', 'request_id': 'req-306937cc-e1c3-477f-9731-590ddbf7fd5f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:53.604160 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-306937cc-e1c3-477f-9731-590ddbf7fd5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c06e827e8c44f58a2051705a46e361c', 'request_id': 'req-306937cc-e1c3-477f-9731-590ddbf7fd5f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:53.616030 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-306937cc-e1c3-477f-9731-590ddbf7fd5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c06e827e8c44f58a2051705a46e361c', 'request_id': 'req-306937cc-e1c3-477f-9731-590ddbf7fd5f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:53.628805 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-306937cc-e1c3-477f-9731-590ddbf7fd5f admin admin] The token cache is being invalidate because project 9c06e827e8c44f58a2051705a46e361c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:53.629140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-306937cc-e1c3-477f-9731-590ddbf7fd5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:53.630118 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-306937cc-e1c3-477f-9731-590ddbf7fd5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:53.635708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1227/2421] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:53 2022] DELETE /identity/v3/projects/9c06e827e8c44f58a2051705a46e361c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:53.642903 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029824d0-c849-44a7-815a-5657268b58e7 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:53.643198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029824d0-c849-44a7-815a-5657268b58e7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:53.643477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029824d0-c849-44a7-815a-5657268b58e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:53.660899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-029824d0-c849-44a7-815a-5657268b58e7 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:53.662111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-029824d0-c849-44a7-815a-5657268b58e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:53.720216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1195/2422] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:22:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:53.728523 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cb1da-c8a9-421c-b15a-f6b518bff5dc None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:53.729029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cb1da-c8a9-421c-b15a-f6b518bff5dc None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:53.729029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cb1da-c8a9-421c-b15a-f6b518bff5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:53.747961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-fe7cb1da-c8a9-421c-b15a-f6b518bff5dc None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:53.749184 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fe7cb1da-c8a9-421c-b15a-f6b518bff5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:53.757806 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1228/2423] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:22:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:53.763162 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029824d0-c849-44a7-815a-5657268b58e7 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:53.774934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029824d0-c849-44a7-815a-5657268b58e7 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:53.780905 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358dd28-a1cb-453b-88b7-3e7bbadc2042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:53.781613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358dd28-a1cb-453b-88b7-3e7bbadc2042 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:53.781877 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358dd28-a1cb-453b-88b7-3e7bbadc2042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:53.782147 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358dd28-a1cb-453b-88b7-3e7bbadc2042 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:53.782881 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358dd28-a1cb-453b-88b7-3e7bbadc2042 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:22:53.783215 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358dd28-a1cb-453b-88b7-3e7bbadc2042 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:53.785167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358dd28-a1cb-453b-88b7-3e7bbadc2042 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:53.786869 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:53.798827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1196/2424] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:22:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:22:53.803238 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7cb1da-c8a9-421c-b15a-f6b518bff5dc None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:53.813083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7cb1da-c8a9-421c-b15a-f6b518bff5dc None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:53.818269 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d81a1d-b545-400b-b9d7-44e50ae244f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:53.819003 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d81a1d-b545-400b-b9d7-44e50ae244f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:53.819358 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d81a1d-b545-400b-b9d7-44e50ae244f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:53.819745 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d81a1d-b545-400b-b9d7-44e50ae244f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:53.820513 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d81a1d-b545-400b-b9d7-44e50ae244f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:53.822501 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d81a1d-b545-400b-b9d7-44e50ae244f0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:53.834163 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:53.871520 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1229/2425] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:53 2022] POST /identity/v3/projects => generated 381 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:53.872607 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358dd28-a1cb-453b-88b7-3e7bbadc2042 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:53.883757 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358dd28-a1cb-453b-88b7-3e7bbadc2042 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:53.942951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61da32-a442-4f6a-8f77-291bfc84dc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:53.943662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61da32-a442-4f6a-8f77-291bfc84dc69 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:53.944095 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61da32-a442-4f6a-8f77-291bfc84dc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:53.944379 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61da32-a442-4f6a-8f77-291bfc84dc69 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:53.945258 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61da32-a442-4f6a-8f77-291bfc84dc69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:53.947255 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61da32-a442-4f6a-8f77-291bfc84dc69 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:53.970660 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1197/2426] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:53 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:53.974913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d81a1d-b545-400b-b9d7-44e50ae244f0 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:53.987662 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d81a1d-b545-400b-b9d7-44e50ae244f0 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:53.997222 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2a1f2-ceb6-41c4-aed2-882928932e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:53.998106 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2a1f2-ceb6-41c4-aed2-882928932e54 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:53.998541 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2a1f2-ceb6-41c4-aed2-882928932e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:53.998912 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2a1f2-ceb6-41c4-aed2-882928932e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:53.999864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2a1f2-ceb6-41c4-aed2-882928932e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:54.001854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2a1f2-ceb6-41c4-aed2-882928932e54 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:54.014485 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1230/2427] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:54.018922 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61da32-a442-4f6a-8f77-291bfc84dc69 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:54.030987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61da32-a442-4f6a-8f77-291bfc84dc69 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:54.037780 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d7f90-54e0-45d3-9041-28c428a15503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:54.038540 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d7f90-54e0-45d3-9041-28c428a15503 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:54.038874 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d7f90-54e0-45d3-9041-28c428a15503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:54.039192 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d7f90-54e0-45d3-9041-28c428a15503 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:54.039972 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d7f90-54e0-45d3-9041-28c428a15503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:54.041602 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d7f90-54e0-45d3-9041-28c428a15503 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:54.053471 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1198/2428] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:54.057458 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2a1f2-ceb6-41c4-aed2-882928932e54 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:54.068791 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2a1f2-ceb6-41c4-aed2-882928932e54 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:54.073793 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685459c-3746-4c03-9e71-653ad486639a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:54.074845 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685459c-3746-4c03-9e71-653ad486639a admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:54.075126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685459c-3746-4c03-9e71-653ad486639a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:54.075414 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685459c-3746-4c03-9e71-653ad486639a admin admin] PATH_INFO: `/v3/projects/65dadc54ec404e92ae361f597340c0c0/users/dfa47af423d247aeb84a3cb7883af296/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:54.092135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685459c-3746-4c03-9e71-653ad486639a admin admin] RBAC: Authorizing `identity:create_grant(project_id=65dadc54ec404e92ae361f597340c0c0, user_id=dfa47af423d247aeb84a3cb7883af296, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:54.094836 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685459c-3746-4c03-9e71-653ad486639a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:54.110619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1231/2429] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:54 2022] PUT /identity/v3/projects/65dadc54ec404e92ae361f597340c0c0/users/dfa47af423d247aeb84a3cb7883af296/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:54.121763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de1103-bcfb-4d41-9c2d-1ce1dc437c03 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:54.122125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de1103-bcfb-4d41-9c2d-1ce1dc437c03 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:54.122506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de1103-bcfb-4d41-9c2d-1ce1dc437c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:54.152651 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-00de1103-bcfb-4d41-9c2d-1ce1dc437c03 None None] MFA Rules not processed for user `dfa47af423d247aeb84a3cb7883af296`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:54.153905 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-00de1103-bcfb-4d41-9c2d-1ce1dc437c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:54.265698 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1199/2430] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:22:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:54.270203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685459c-3746-4c03-9e71-653ad486639a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:54.411657 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685459c-3746-4c03-9e71-653ad486639a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:54.416189 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9580fc1-da8f-4b4b-a68d-f6a3addb04d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:54.416870 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9580fc1-da8f-4b4b-a68d-f6a3addb04d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:54.423910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9580fc1-da8f-4b4b-a68d-f6a3addb04d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:54.423910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9580fc1-da8f-4b4b-a68d-f6a3addb04d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:54.423910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9580fc1-da8f-4b4b-a68d-f6a3addb04d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:54.423910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9580fc1-da8f-4b4b-a68d-f6a3addb04d5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:54.467068 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1232/2431] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:54 2022] POST /identity/v3/projects => generated 381 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:54.470920 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de1103-bcfb-4d41-9c2d-1ce1dc437c03 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:54.484468 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de1103-bcfb-4d41-9c2d-1ce1dc437c03 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:54.557837 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fca824-a2e1-43dc-807b-817bca6f1773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:54.558599 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fca824-a2e1-43dc-807b-817bca6f1773 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:54.559004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fca824-a2e1-43dc-807b-817bca6f1773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:54.559344 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fca824-a2e1-43dc-807b-817bca6f1773 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:54.560388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fca824-a2e1-43dc-807b-817bca6f1773 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:54.566377 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fca824-a2e1-43dc-807b-817bca6f1773 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:54.577790 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:54.613649 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1200/2432] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:54 2022] POST /identity/v3/users => generated 297 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:54.617666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9580fc1-da8f-4b4b-a68d-f6a3addb04d5 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:54.631254 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9580fc1-da8f-4b4b-a68d-f6a3addb04d5 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:54.641501 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f53d2e-e890-4acc-81ed-1e955bedfb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:54.642505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f53d2e-e890-4acc-81ed-1e955bedfb60 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:54.643438 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f53d2e-e890-4acc-81ed-1e955bedfb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:54.644176 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f53d2e-e890-4acc-81ed-1e955bedfb60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:54.645579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f53d2e-e890-4acc-81ed-1e955bedfb60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:54.651266 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f53d2e-e890-4acc-81ed-1e955bedfb60 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:54.701353 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1233/2433] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:22:54.702438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fca824-a2e1-43dc-807b-817bca6f1773 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:54.716271 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fca824-a2e1-43dc-807b-817bca6f1773 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:54.726204 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec20b88-6f85-45d9-9b3c-63f073aaf78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:54.728592 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec20b88-6f85-45d9-9b3c-63f073aaf78b admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:54.729164 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec20b88-6f85-45d9-9b3c-63f073aaf78b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:54.730063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec20b88-6f85-45d9-9b3c-63f073aaf78b admin admin] PATH_INFO: `/v3/projects/03e5755237374a398e1a45ef64666269/users/9fc6b5ebb9434e2d81c3f94707a444dd/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:54.763692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec20b88-6f85-45d9-9b3c-63f073aaf78b admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e5755237374a398e1a45ef64666269, user_id=9fc6b5ebb9434e2d81c3f94707a444dd, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:54.769279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec20b88-6f85-45d9-9b3c-63f073aaf78b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:54.791951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1201/2434] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:54 2022] PUT /identity/v3/projects/03e5755237374a398e1a45ef64666269/users/9fc6b5ebb9434e2d81c3f94707a444dd/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:54.796503 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f53d2e-e890-4acc-81ed-1e955bedfb60 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:54.819638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f53d2e-e890-4acc-81ed-1e955bedfb60 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:54.904599 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66beff41-3d96-4e0e-adca-d5be80f4cc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:54.905514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beff41-3d96-4e0e-adca-d5be80f4cc4b admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:54.905917 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beff41-3d96-4e0e-adca-d5be80f4cc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:54.906302 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beff41-3d96-4e0e-adca-d5be80f4cc4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:54.907164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66beff41-3d96-4e0e-adca-d5be80f4cc4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:54.908933 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66beff41-3d96-4e0e-adca-d5be80f4cc4b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:54.922272 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1234/2435] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:54.925579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec20b88-6f85-45d9-9b3c-63f073aaf78b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:54.942193 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec20b88-6f85-45d9-9b3c-63f073aaf78b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:54.947328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92892e0e-5bd0-49ab-b2f4-6d39eef5d576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:54.948370 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92892e0e-5bd0-49ab-b2f4-6d39eef5d576 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:54.948653 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92892e0e-5bd0-49ab-b2f4-6d39eef5d576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:54.948930 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92892e0e-5bd0-49ab-b2f4-6d39eef5d576 admin admin] PATH_INFO: `/v3/projects/03e5755237374a398e1a45ef64666269/users/9fc6b5ebb9434e2d81c3f94707a444dd/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:54.951010 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92892e0e-5bd0-49ab-b2f4-6d39eef5d576 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e5755237374a398e1a45ef64666269, user_id=9fc6b5ebb9434e2d81c3f94707a444dd, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:54.953743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92892e0e-5bd0-49ab-b2f4-6d39eef5d576 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:54.994290 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1202/2436] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:54 2022] PUT /identity/v3/projects/03e5755237374a398e1a45ef64666269/users/9fc6b5ebb9434e2d81c3f94707a444dd/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:54.996273 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76925e4c-2fa9-4d52-b400-8d0972a8cee3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:54.996572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76925e4c-2fa9-4d52-b400-8d0972a8cee3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:54.996850 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76925e4c-2fa9-4d52-b400-8d0972a8cee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:55.024608 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-76925e4c-2fa9-4d52-b400-8d0972a8cee3 None None] MFA Rules not processed for user `9fc6b5ebb9434e2d81c3f94707a444dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:55.025838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-76925e4c-2fa9-4d52-b400-8d0972a8cee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:55.030983 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:55.096364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1235/2437] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:22:54 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:55.107600 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92892e0e-5bd0-49ab-b2f4-6d39eef5d576 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:55.109460 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-92892e0e-5bd0-49ab-b2f4-6d39eef5d576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:55.183224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92892e0e-5bd0-49ab-b2f4-6d39eef5d576 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:55.186973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0f087-d9e6-4ede-93e0-f5ddfb41b525 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:55.187663 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0f087-d9e6-4ede-93e0-f5ddfb41b525 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:55.187954 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0f087-d9e6-4ede-93e0-f5ddfb41b525 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:55.188231 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0f087-d9e6-4ede-93e0-f5ddfb41b525 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:55.197707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0f087-d9e6-4ede-93e0-f5ddfb41b525 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:55.199178 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0f087-d9e6-4ede-93e0-f5ddfb41b525 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:55.209374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1203/2438] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:55.283580 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76925e4c-2fa9-4d52-b400-8d0972a8cee3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:55.293630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76925e4c-2fa9-4d52-b400-8d0972a8cee3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:55.297610 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8bb7f-8002-4c4e-94cd-1f0bfb0f52be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:55.298291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8bb7f-8002-4c4e-94cd-1f0bfb0f52be service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:55.298572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8bb7f-8002-4c4e-94cd-1f0bfb0f52be service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:55.298846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8bb7f-8002-4c4e-94cd-1f0bfb0f52be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:55.360965 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8bb7f-8002-4c4e-94cd-1f0bfb0f52be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:55.362434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8bb7f-8002-4c4e-94cd-1f0bfb0f52be service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:55.371849 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1236/2439] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:56.700423 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5fbec-6cf8-446d-8314-4fa9d33bf98c None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:56.701309 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5fbec-6cf8-446d-8314-4fa9d33bf98c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:56.701749 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5fbec-6cf8-446d-8314-4fa9d33bf98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:56.725204 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d9d5fbec-6cf8-446d-8314-4fa9d33bf98c None None] MFA Rules not processed for user `e802da0b078e457da466a9a7b0f15085`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:56.729554 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d9d5fbec-6cf8-446d-8314-4fa9d33bf98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:56.868332 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1204/2440] 10.222.0.27 () {58 vars in 967 bytes} [Mon Jun 27 21:22:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:56.896814 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8bb7f-8002-4c4e-94cd-1f0bfb0f52be service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:56.899234 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-64a8bb7f-8002-4c4e-94cd-1f0bfb0f52be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:56.990895 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8bb7f-8002-4c4e-94cd-1f0bfb0f52be service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:56.997981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05cf44d-cd00-4989-9e42-24a337785119 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28acb0322114aebafcf62bdf9a346df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:56.998705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05cf44d-cd00-4989-9e42-24a337785119 service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:56.998997 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05cf44d-cd00-4989-9e42-24a337785119 service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:56.999428 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05cf44d-cd00-4989-9e42-24a337785119 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:57.009925 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05cf44d-cd00-4989-9e42-24a337785119 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:57.012390 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05cf44d-cd00-4989-9e42-24a337785119 service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:57.023784 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1237/2441] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Jun 27 21:22:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 21:23:02.578197 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5fbec-6cf8-446d-8314-4fa9d33bf98c None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:02.579775 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d9d5fbec-6cf8-446d-8314-4fa9d33bf98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:02.640646 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5fbec-6cf8-446d-8314-4fa9d33bf98c None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:02.644860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad207e60-2576-4332-b1cf-fa3e79d2edb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:02.645518 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad207e60-2576-4332-b1cf-fa3e79d2edb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:02.645797 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad207e60-2576-4332-b1cf-fa3e79d2edb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:02.646070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad207e60-2576-4332-b1cf-fa3e79d2edb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:02.648397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ad207e60-2576-4332-b1cf-fa3e79d2edb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:02.654050 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:02.718225 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad207e60-2576-4332-b1cf-fa3e79d2edb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:02.719774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad207e60-2576-4332-b1cf-fa3e79d2edb3 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:02.729912 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1205/2442] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:05.598363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05cf44d-cd00-4989-9e42-24a337785119 service swift] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:05.612548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05cf44d-cd00-4989-9e42-24a337785119 service swift] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:05.619969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544192b-40f4-4666-a23b-83a6c200574f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:05.621003 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544192b-40f4-4666-a23b-83a6c200574f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:05.621980 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544192b-40f4-4666-a23b-83a6c200574f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:05.621980 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544192b-40f4-4666-a23b-83a6c200574f admin admin] PATH_INFO: `/v3/users/dfa47af423d247aeb84a3cb7883af296` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:05.623166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544192b-40f4-4666-a23b-83a6c200574f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfa47af423d247aeb84a3cb7883af296)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:05.625515 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544192b-40f4-4666-a23b-83a6c200574f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:05.668889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:05.700970 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a544192b-40f4-4666-a23b-83a6c200574f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfa47af423d247aeb84a3cb7883af296'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:05.701752 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a544192b-40f4-4666-a23b-83a6c200574f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfa47af423d247aeb84a3cb7883af296'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:05.702509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a544192b-40f4-4666-a23b-83a6c200574f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfa47af423d247aeb84a3cb7883af296'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:05.719298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a544192b-40f4-4666-a23b-83a6c200574f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfa47af423d247aeb84a3cb7883af296'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:05.729883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a544192b-40f4-4666-a23b-83a6c200574f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfa47af423d247aeb84a3cb7883af296'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:05.754543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1238/2443] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:05 2022] DELETE /identity/v3/users/dfa47af423d247aeb84a3cb7883af296 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:05.757886 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e10db-333b-419b-bc0a-ff23fe0ad212 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:05.758390 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e10db-333b-419b-bc0a-ff23fe0ad212 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:05.758475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e10db-333b-419b-bc0a-ff23fe0ad212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:05.784359 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-4c6e10db-333b-419b-bc0a-ff23fe0ad212 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:05.785668 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4c6e10db-333b-419b-bc0a-ff23fe0ad212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:05.869774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1206/2444] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:05.880469 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544192b-40f4-4666-a23b-83a6c200574f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:05.881978 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a544192b-40f4-4666-a23b-83a6c200574f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:05.944749 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544192b-40f4-4666-a23b-83a6c200574f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:05.948569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30015b3-212c-4753-967a-6254a9e8746c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:05.949218 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30015b3-212c-4753-967a-6254a9e8746c service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:05.949498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30015b3-212c-4753-967a-6254a9e8746c service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:05.949767 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30015b3-212c-4753-967a-6254a9e8746c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:05.964702 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30015b3-212c-4753-967a-6254a9e8746c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:05.966203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30015b3-212c-4753-967a-6254a9e8746c service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:05.983758 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1239/2445] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:06.223210 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e10db-333b-419b-bc0a-ff23fe0ad212 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:06.224176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4c6e10db-333b-419b-bc0a-ff23fe0ad212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:06.236565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e10db-333b-419b-bc0a-ff23fe0ad212 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:06.241523 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690c659-908f-42cd-bbee-4ffbf96f0915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:06.242406 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690c659-908f-42cd-bbee-4ffbf96f0915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:06.242686 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690c659-908f-42cd-bbee-4ffbf96f0915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:06.243659 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690c659-908f-42cd-bbee-4ffbf96f0915 admin admin] PATH_INFO: `/v3/users/30cbde26a1964f22902c24917006fb90` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:06.245087 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690c659-908f-42cd-bbee-4ffbf96f0915 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30cbde26a1964f22902c24917006fb90)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:06.247115 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690c659-908f-42cd-bbee-4ffbf96f0915 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:06.295970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a690c659-908f-42cd-bbee-4ffbf96f0915 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30cbde26a1964f22902c24917006fb90'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:06.296620 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a690c659-908f-42cd-bbee-4ffbf96f0915 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30cbde26a1964f22902c24917006fb90'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:06.297219 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a690c659-908f-42cd-bbee-4ffbf96f0915 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30cbde26a1964f22902c24917006fb90'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:06.311870 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a690c659-908f-42cd-bbee-4ffbf96f0915 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30cbde26a1964f22902c24917006fb90'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:06.319260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a690c659-908f-42cd-bbee-4ffbf96f0915 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30cbde26a1964f22902c24917006fb90'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:06.333674 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1207/2446] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:06 2022] DELETE /identity/v3/users/30cbde26a1964f22902c24917006fb90 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:06.344323 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30015b3-212c-4753-967a-6254a9e8746c service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:06.346006 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c30015b3-212c-4753-967a-6254a9e8746c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:06.367852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30015b3-212c-4753-967a-6254a9e8746c service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:06.431736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408c2c4-62f5-44a8-84e6-ec562d000e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:06.432619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408c2c4-62f5-44a8-84e6-ec562d000e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:06.432902 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408c2c4-62f5-44a8-84e6-ec562d000e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:06.433218 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408c2c4-62f5-44a8-84e6-ec562d000e1e admin admin] PATH_INFO: `/v3/projects/65dadc54ec404e92ae361f597340c0c0` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:06.434419 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408c2c4-62f5-44a8-84e6-ec562d000e1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=65dadc54ec404e92ae361f597340c0c0)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:06.436385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408c2c4-62f5-44a8-84e6-ec562d000e1e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:06.484573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:06.485863 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c408c2c4-62f5-44a8-84e6-ec562d000e1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65dadc54ec404e92ae361f597340c0c0', 'request_id': 'req-c408c2c4-62f5-44a8-84e6-ec562d000e1e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:06.495038 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c408c2c4-62f5-44a8-84e6-ec562d000e1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65dadc54ec404e92ae361f597340c0c0', 'request_id': 'req-c408c2c4-62f5-44a8-84e6-ec562d000e1e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:06.503871 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c408c2c4-62f5-44a8-84e6-ec562d000e1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65dadc54ec404e92ae361f597340c0c0', 'request_id': 'req-c408c2c4-62f5-44a8-84e6-ec562d000e1e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:06.513999 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690c659-908f-42cd-bbee-4ffbf96f0915 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:06.515798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a690c659-908f-42cd-bbee-4ffbf96f0915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:06.516036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c408c2c4-62f5-44a8-84e6-ec562d000e1e admin admin] The token cache is being invalidate because project 65dadc54ec404e92ae361f597340c0c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:06.516329 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c408c2c4-62f5-44a8-84e6-ec562d000e1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:06.516973 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c408c2c4-62f5-44a8-84e6-ec562d000e1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:06.518617 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1240/2447] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:06 2022] DELETE /identity/v3/projects/65dadc54ec404e92ae361f597340c0c0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:06.522426 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408c2c4-62f5-44a8-84e6-ec562d000e1e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:06.523976 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c408c2c4-62f5-44a8-84e6-ec562d000e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:06.542594 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690c659-908f-42cd-bbee-4ffbf96f0915 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:06.552658 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:06.596410 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408c2c4-62f5-44a8-84e6-ec562d000e1e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:06.601254 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e10d04-93e1-47a0-80ec-8855bed3e00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:06.602203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e10d04-93e1-47a0-80ec-8855bed3e00a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:06.602615 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e10d04-93e1-47a0-80ec-8855bed3e00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:06.603009 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e10d04-93e1-47a0-80ec-8855bed3e00a admin admin] PATH_INFO: `/v3/users/9fc6b5ebb9434e2d81c3f94707a444dd` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:06.604151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e10d04-93e1-47a0-80ec-8855bed3e00a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fc6b5ebb9434e2d81c3f94707a444dd)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:06.606173 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e10d04-93e1-47a0-80ec-8855bed3e00a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:06.608241 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95d1e8-c221-4884-8d7d-99cbc8f6da7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:06.609207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95d1e8-c221-4884-8d7d-99cbc8f6da7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:06.609484 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95d1e8-c221-4884-8d7d-99cbc8f6da7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:06.609757 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95d1e8-c221-4884-8d7d-99cbc8f6da7d admin admin] PATH_INFO: `/v3/projects/0b5921bd9539401d9a235ccde185632b` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:06.611044 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95d1e8-c221-4884-8d7d-99cbc8f6da7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b5921bd9539401d9a235ccde185632b)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:06.613206 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95d1e8-c221-4884-8d7d-99cbc8f6da7d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:06.654496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-96e10d04-93e1-47a0-80ec-8855bed3e00a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fc6b5ebb9434e2d81c3f94707a444dd'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:06.655142 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-96e10d04-93e1-47a0-80ec-8855bed3e00a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fc6b5ebb9434e2d81c3f94707a444dd'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:06.655846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-96e10d04-93e1-47a0-80ec-8855bed3e00a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fc6b5ebb9434e2d81c3f94707a444dd'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:06.661925 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-df95d1e8-c221-4884-8d7d-99cbc8f6da7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b5921bd9539401d9a235ccde185632b', 'request_id': 'req-df95d1e8-c221-4884-8d7d-99cbc8f6da7d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:06.669282 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-96e10d04-93e1-47a0-80ec-8855bed3e00a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fc6b5ebb9434e2d81c3f94707a444dd'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:06.670305 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-df95d1e8-c221-4884-8d7d-99cbc8f6da7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b5921bd9539401d9a235ccde185632b', 'request_id': 'req-df95d1e8-c221-4884-8d7d-99cbc8f6da7d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:06.676461 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-96e10d04-93e1-47a0-80ec-8855bed3e00a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fc6b5ebb9434e2d81c3f94707a444dd'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:06.676607 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-df95d1e8-c221-4884-8d7d-99cbc8f6da7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b5921bd9539401d9a235ccde185632b', 'request_id': 'req-df95d1e8-c221-4884-8d7d-99cbc8f6da7d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:06.688134 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-df95d1e8-c221-4884-8d7d-99cbc8f6da7d admin admin] The token cache is being invalidate because project 0b5921bd9539401d9a235ccde185632b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:06.688350 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-df95d1e8-c221-4884-8d7d-99cbc8f6da7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:06.689008 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-df95d1e8-c221-4884-8d7d-99cbc8f6da7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:06.689883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1241/2448] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:06 2022] DELETE /identity/v3/users/9fc6b5ebb9434e2d81c3f94707a444dd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:06.690657 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1208/2449] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:06 2022] DELETE /identity/v3/projects/0b5921bd9539401d9a235ccde185632b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:06.695676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2893f62-ed89-4133-8e20-d5be73e7ead6 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:06.695988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2893f62-ed89-4133-8e20-d5be73e7ead6 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:06.696257 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2893f62-ed89-4133-8e20-d5be73e7ead6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:06.713624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b2893f62-ed89-4133-8e20-d5be73e7ead6 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:06.715003 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b2893f62-ed89-4133-8e20-d5be73e7ead6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:06.767400 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:06.782542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1242/2450] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:06.791074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300ff57-b728-4757-a849-f22fbb27f558 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:06.791526 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300ff57-b728-4757-a849-f22fbb27f558 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:06.791726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300ff57-b728-4757-a849-f22fbb27f558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:06.825017 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b300ff57-b728-4757-a849-f22fbb27f558 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:06.829134 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b300ff57-b728-4757-a849-f22fbb27f558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:06.843567 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1209/2451] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:06.847876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2893f62-ed89-4133-8e20-d5be73e7ead6 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:06.860227 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2893f62-ed89-4133-8e20-d5be73e7ead6 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:06.866238 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fdce7a-6a6b-45cf-91ca-04a0ca4c4e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:06.867154 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdce7a-6a6b-45cf-91ca-04a0ca4c4e8d admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:06.867591 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdce7a-6a6b-45cf-91ca-04a0ca4c4e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:06.867913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdce7a-6a6b-45cf-91ca-04a0ca4c4e8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:06.868968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fdce7a-6a6b-45cf-91ca-04a0ca4c4e8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:23:06.868968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fdce7a-6a6b-45cf-91ca-04a0ca4c4e8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:06.870610 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fdce7a-6a6b-45cf-91ca-04a0ca4c4e8d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:06.886697 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1243/2452] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:23:06.891874 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300ff57-b728-4757-a849-f22fbb27f558 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:06.902127 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300ff57-b728-4757-a849-f22fbb27f558 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:06.912198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df3360-d6db-474f-8dda-ae52558773f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:06.913595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df3360-d6db-474f-8dda-ae52558773f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:06.914093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df3360-d6db-474f-8dda-ae52558773f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:06.914556 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df3360-d6db-474f-8dda-ae52558773f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:06.915719 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03df3360-d6db-474f-8dda-ae52558773f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:06.918174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03df3360-d6db-474f-8dda-ae52558773f6 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:06.953880 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1210/2453] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:06 2022] POST /identity/v3/projects => generated 405 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:06.957749 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fdce7a-6a6b-45cf-91ca-04a0ca4c4e8d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:06.970684 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fdce7a-6a6b-45cf-91ca-04a0ca4c4e8d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:07.043209 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6cd223-4d90-41d6-b9f7-bfe3ae00ebe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:07.043926 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6cd223-4d90-41d6-b9f7-bfe3ae00ebe9 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:07.044208 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6cd223-4d90-41d6-b9f7-bfe3ae00ebe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:07.044476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6cd223-4d90-41d6-b9f7-bfe3ae00ebe9 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:07.045179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6cd223-4d90-41d6-b9f7-bfe3ae00ebe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:07.047126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6cd223-4d90-41d6-b9f7-bfe3ae00ebe9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:07.068349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1244/2454] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:06 2022] POST /identity/v3/users => generated 303 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:07.072563 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df3360-d6db-474f-8dda-ae52558773f6 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:07.084224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df3360-d6db-474f-8dda-ae52558773f6 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:07.089772 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc9263-eb80-4f7d-91bd-cccfe5e44d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:07.090631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc9263-eb80-4f7d-91bd-cccfe5e44d2c admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:07.090960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc9263-eb80-4f7d-91bd-cccfe5e44d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:07.091243 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc9263-eb80-4f7d-91bd-cccfe5e44d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:07.092021 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc9263-eb80-4f7d-91bd-cccfe5e44d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:07.093652 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc9263-eb80-4f7d-91bd-cccfe5e44d2c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:07.105579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1211/2455] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:23:07.109512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6cd223-4d90-41d6-b9f7-bfe3ae00ebe9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:07.120151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6cd223-4d90-41d6-b9f7-bfe3ae00ebe9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:07.125482 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b2c1c-85cd-4e70-9015-ae27642fa2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:07.126208 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b2c1c-85cd-4e70-9015-ae27642fa2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:07.126496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b2c1c-85cd-4e70-9015-ae27642fa2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:07.126770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b2c1c-85cd-4e70-9015-ae27642fa2f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:07.127741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b2c1c-85cd-4e70-9015-ae27642fa2f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:07.129765 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b2c1c-85cd-4e70-9015-ae27642fa2f2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:07.141304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1245/2456] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:07.145876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc9263-eb80-4f7d-91bd-cccfe5e44d2c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:07.156375 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc9263-eb80-4f7d-91bd-cccfe5e44d2c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:07.167270 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fc937-3ed5-41c2-9fb3-e970a9ff356e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:07.168220 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fc937-3ed5-41c2-9fb3-e970a9ff356e admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:07.168501 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fc937-3ed5-41c2-9fb3-e970a9ff356e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:07.168759 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fc937-3ed5-41c2-9fb3-e970a9ff356e admin admin] PATH_INFO: `/v3/projects/828cf440c5dc4d7ab5da4b72b8846eea/users/cfd454e5ea9b415e9042199fa64db2d4/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:07.186448 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fc937-3ed5-41c2-9fb3-e970a9ff356e admin admin] RBAC: Authorizing `identity:create_grant(project_id=828cf440c5dc4d7ab5da4b72b8846eea, user_id=cfd454e5ea9b415e9042199fa64db2d4, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:07.189382 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fc937-3ed5-41c2-9fb3-e970a9ff356e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:07.205105 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1212/2457] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:07 2022] PUT /identity/v3/projects/828cf440c5dc4d7ab5da4b72b8846eea/users/cfd454e5ea9b415e9042199fa64db2d4/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:07.210367 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d111e6-793c-48b1-8893-ae44408d9f4c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:07.210684 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d111e6-793c-48b1-8893-ae44408d9f4c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:07.210962 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d111e6-793c-48b1-8893-ae44408d9f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:07.232367 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-05d111e6-793c-48b1-8893-ae44408d9f4c None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:07.233761 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-05d111e6-793c-48b1-8893-ae44408d9f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:07.300533 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1246/2458] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:07.305890 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fc937-3ed5-41c2-9fb3-e970a9ff356e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:07.307626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-548fc937-3ed5-41c2-9fb3-e970a9ff356e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:07.311182 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d111e6-793c-48b1-8893-ae44408d9f4c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:07.313864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-05d111e6-793c-48b1-8893-ae44408d9f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:07.320900 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fc937-3ed5-41c2-9fb3-e970a9ff356e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:07.328354 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e57d9-3698-4eee-ae69-4a86d4757107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:07.329163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e57d9-3698-4eee-ae69-4a86d4757107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:07.329865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e57d9-3698-4eee-ae69-4a86d4757107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:07.330147 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e57d9-3698-4eee-ae69-4a86d4757107 admin admin] PATH_INFO: `/v3/projects/03e5755237374a398e1a45ef64666269` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:07.331776 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e57d9-3698-4eee-ae69-4a86d4757107 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03e5755237374a398e1a45ef64666269)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:07.333798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e57d9-3698-4eee-ae69-4a86d4757107 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:07.382804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:07.394257 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d111e6-793c-48b1-8893-ae44408d9f4c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:07.395800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9f7e57d9-3698-4eee-ae69-4a86d4757107 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03e5755237374a398e1a45ef64666269', 'request_id': 'req-9f7e57d9-3698-4eee-ae69-4a86d4757107'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:07.404664 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9f7e57d9-3698-4eee-ae69-4a86d4757107 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03e5755237374a398e1a45ef64666269', 'request_id': 'req-9f7e57d9-3698-4eee-ae69-4a86d4757107'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:07.408277 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:07.411810 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9f7e57d9-3698-4eee-ae69-4a86d4757107 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03e5755237374a398e1a45ef64666269', 'request_id': 'req-9f7e57d9-3698-4eee-ae69-4a86d4757107'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:07.430236 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9f7e57d9-3698-4eee-ae69-4a86d4757107 admin admin] The token cache is being invalidate because project 03e5755237374a398e1a45ef64666269 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:07.430583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9f7e57d9-3698-4eee-ae69-4a86d4757107 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:07.431484 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9f7e57d9-3698-4eee-ae69-4a86d4757107 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:07.437894 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1213/2459] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:07 2022] DELETE /identity/v3/projects/03e5755237374a398e1a45ef64666269 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:07.439547 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f222532-64c2-4606-814f-92907fc5dc30 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:07.439799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f222532-64c2-4606-814f-92907fc5dc30 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:07.440078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f222532-64c2-4606-814f-92907fc5dc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:07.451811 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab2680-dd55-42cd-8960-38f19155b76a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:07.452127 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab2680-dd55-42cd-8960-38f19155b76a service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:07.452413 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab2680-dd55-42cd-8960-38f19155b76a service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:07.452692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab2680-dd55-42cd-8960-38f19155b76a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:07.454567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d9ab2680-dd55-42cd-8960-38f19155b76a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:07.461104 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-8f222532-64c2-4606-814f-92907fc5dc30 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:07.462414 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8f222532-64c2-4606-814f-92907fc5dc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:07.520492 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab2680-dd55-42cd-8960-38f19155b76a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:07.521944 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab2680-dd55-42cd-8960-38f19155b76a service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:07.531885 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1247/2460] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:07.533215 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1214/2461] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:07.540807 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687617f-1b93-4b56-89e5-75f7b3b6f67d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:07.541194 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687617f-1b93-4b56-89e5-75f7b3b6f67d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:07.541518 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687617f-1b93-4b56-89e5-75f7b3b6f67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:07.564124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f687617f-1b93-4b56-89e5-75f7b3b6f67d None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:07.565509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f687617f-1b93-4b56-89e5-75f7b3b6f67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:07.576894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1248/2462] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:07.577887 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f222532-64c2-4606-814f-92907fc5dc30 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:07.587872 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f222532-64c2-4606-814f-92907fc5dc30 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:07.593185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88221821-ae4f-4dc0-ab90-d7ef644cf412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:07.593916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88221821-ae4f-4dc0-ab90-d7ef644cf412 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:07.594202 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88221821-ae4f-4dc0-ab90-d7ef644cf412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:07.594774 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88221821-ae4f-4dc0-ab90-d7ef644cf412 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:07.595349 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88221821-ae4f-4dc0-ab90-d7ef644cf412 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:23:07.595696 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88221821-ae4f-4dc0-ab90-d7ef644cf412 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:07.597437 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88221821-ae4f-4dc0-ab90-d7ef644cf412 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:07.615833 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1215/2463] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:23:07.617048 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687617f-1b93-4b56-89e5-75f7b3b6f67d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:07.631013 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687617f-1b93-4b56-89e5-75f7b3b6f67d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:07.639055 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd100e9b-6ad0-42b2-8e5c-11a9459b2133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:07.640150 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd100e9b-6ad0-42b2-8e5c-11a9459b2133 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:07.640441 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd100e9b-6ad0-42b2-8e5c-11a9459b2133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:07.640719 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd100e9b-6ad0-42b2-8e5c-11a9459b2133 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:07.641458 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd100e9b-6ad0-42b2-8e5c-11a9459b2133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:07.643946 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd100e9b-6ad0-42b2-8e5c-11a9459b2133 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:07.679664 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1249/2464] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:07 2022] POST /identity/v3/projects => generated 387 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:07.681622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88221821-ae4f-4dc0-ab90-d7ef644cf412 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:07.692476 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88221821-ae4f-4dc0-ab90-d7ef644cf412 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:07.762793 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114315c-1b81-4e4f-8c05-38a3f77f519e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:07.763540 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114315c-1b81-4e4f-8c05-38a3f77f519e admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:07.763850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114315c-1b81-4e4f-8c05-38a3f77f519e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:07.764126 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114315c-1b81-4e4f-8c05-38a3f77f519e admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:07.764820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114315c-1b81-4e4f-8c05-38a3f77f519e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:07.766908 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114315c-1b81-4e4f-8c05-38a3f77f519e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:07.795042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1216/2465] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:07 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:07.797183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd100e9b-6ad0-42b2-8e5c-11a9459b2133 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:07.810045 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd100e9b-6ad0-42b2-8e5c-11a9459b2133 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:07.814904 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf25ddc-1f2f-40ef-ab5a-9ae6423f6b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:07.815578 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf25ddc-1f2f-40ef-ab5a-9ae6423f6b9c admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:07.815893 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf25ddc-1f2f-40ef-ab5a-9ae6423f6b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:07.816470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf25ddc-1f2f-40ef-ab5a-9ae6423f6b9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:07.817011 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf25ddc-1f2f-40ef-ab5a-9ae6423f6b9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:07.818623 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf25ddc-1f2f-40ef-ab5a-9ae6423f6b9c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:07.829532 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1250/2466] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:07.834700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114315c-1b81-4e4f-8c05-38a3f77f519e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:07.858274 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114315c-1b81-4e4f-8c05-38a3f77f519e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:07.862984 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed3774-a8b0-4cf0-b1e2-8438570c9fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:07.863713 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed3774-a8b0-4cf0-b1e2-8438570c9fca admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:07.863994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed3774-a8b0-4cf0-b1e2-8438570c9fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:07.864267 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed3774-a8b0-4cf0-b1e2-8438570c9fca admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:07.864990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed3774-a8b0-4cf0-b1e2-8438570c9fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:07.866591 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed3774-a8b0-4cf0-b1e2-8438570c9fca admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:07.877345 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1217/2467] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:07.882509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf25ddc-1f2f-40ef-ab5a-9ae6423f6b9c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:07.893408 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf25ddc-1f2f-40ef-ab5a-9ae6423f6b9c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:07.899484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76a0ed-9e41-470a-874f-c50508cddf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:07.900459 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76a0ed-9e41-470a-874f-c50508cddf35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:07.900891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76a0ed-9e41-470a-874f-c50508cddf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:07.901168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76a0ed-9e41-470a-874f-c50508cddf35 admin admin] PATH_INFO: `/v3/projects/d9dbb9c7a0e74808b1513ee1702e4279/users/4528c3346c414dc3bfe9c1638aa83f75/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:07.918500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d76a0ed-9e41-470a-874f-c50508cddf35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9dbb9c7a0e74808b1513ee1702e4279, user_id=4528c3346c414dc3bfe9c1638aa83f75, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:07.921517 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d76a0ed-9e41-470a-874f-c50508cddf35 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:07.937048 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1251/2468] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:07 2022] PUT /identity/v3/projects/d9dbb9c7a0e74808b1513ee1702e4279/users/4528c3346c414dc3bfe9c1638aa83f75/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:07.949362 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d41784-1c65-4951-8ac1-0ac559260075 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:07.950054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d41784-1c65-4951-8ac1-0ac559260075 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:07.950184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d41784-1c65-4951-8ac1-0ac559260075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:07.980473 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-22d41784-1c65-4951-8ac1-0ac559260075 None None] MFA Rules not processed for user `4528c3346c414dc3bfe9c1638aa83f75`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:07.982061 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-22d41784-1c65-4951-8ac1-0ac559260075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:08.004766 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:08.054146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1218/2469] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:08.058834 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76a0ed-9e41-470a-874f-c50508cddf35 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:08.125340 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76a0ed-9e41-470a-874f-c50508cddf35 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:08.130783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5add8f4-6f91-41f5-9c78-c09ae4ae4aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:08.131455 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5add8f4-6f91-41f5-9c78-c09ae4ae4aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:08.131791 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5add8f4-6f91-41f5-9c78-c09ae4ae4aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:08.132070 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5add8f4-6f91-41f5-9c78-c09ae4ae4aa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:08.132767 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5add8f4-6f91-41f5-9c78-c09ae4ae4aa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:08.134699 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5add8f4-6f91-41f5-9c78-c09ae4ae4aa0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:08.143968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:08.169023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1252/2470] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:08 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:08.173419 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d41784-1c65-4951-8ac1-0ac559260075 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:08.190681 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d41784-1c65-4951-8ac1-0ac559260075 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:08.217833 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b42ac8-d4a8-4e08-ab05-74963c55f32a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:08.218448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b42ac8-d4a8-4e08-ab05-74963c55f32a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:08.218824 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b42ac8-d4a8-4e08-ab05-74963c55f32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:08.249664 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a5b42ac8-d4a8-4e08-ab05-74963c55f32a None None] MFA Rules not processed for user `cfd454e5ea9b415e9042199fa64db2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:08.250684 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a5b42ac8-d4a8-4e08-ab05-74963c55f32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:08.277487 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fcfa76-10c6-44b4-9bb9-66e4a4533d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:08.278174 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcfa76-10c6-44b4-9bb9-66e4a4533d23 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:08.278463 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcfa76-10c6-44b4-9bb9-66e4a4533d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:08.278860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcfa76-10c6-44b4-9bb9-66e4a4533d23 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:08.279563 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fcfa76-10c6-44b4-9bb9-66e4a4533d23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:08.281529 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fcfa76-10c6-44b4-9bb9-66e4a4533d23 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:08.311728 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1219/2471] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:08 2022] POST /identity/v3/users => generated 300 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:08.315751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fcfa76-10c6-44b4-9bb9-66e4a4533d23 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:08.330363 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fcfa76-10c6-44b4-9bb9-66e4a4533d23 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:08.331815 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1253/2472] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:08 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:08.333489 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b42ac8-d4a8-4e08-ab05-74963c55f32a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:08.335161 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a5b42ac8-d4a8-4e08-ab05-74963c55f32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:08.336518 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4ab10-6f29-44cb-9387-5bd2d754f01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:08.337317 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4ab10-6f29-44cb-9387-5bd2d754f01e admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:08.337679 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4ab10-6f29-44cb-9387-5bd2d754f01e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:08.338029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4ab10-6f29-44cb-9387-5bd2d754f01e admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:08.338970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4ab10-6f29-44cb-9387-5bd2d754f01e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:08.340870 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4ab10-6f29-44cb-9387-5bd2d754f01e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:08.350931 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b42ac8-d4a8-4e08-ab05-74963c55f32a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:08.355594 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf8c20-7671-4caf-ac0c-1a6e37145edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:08.356384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf8c20-7671-4caf-ac0c-1a6e37145edd admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:08.356579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1220/2473] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:23:08.356686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf8c20-7671-4caf-ac0c-1a6e37145edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:08.356932 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf8c20-7671-4caf-ac0c-1a6e37145edd admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:08.357619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf8c20-7671-4caf-ac0c-1a6e37145edd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:08.357734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4ab10-6f29-44cb-9387-5bd2d754f01e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:08.359738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf8c20-7671-4caf-ac0c-1a6e37145edd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:08.367875 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4ab10-6f29-44cb-9387-5bd2d754f01e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:08.374817 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7537e-ebe1-4436-a423-8d10aeee6365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:08.375752 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7537e-ebe1-4436-a423-8d10aeee6365 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:08.376034 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7537e-ebe1-4436-a423-8d10aeee6365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:08.376309 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7537e-ebe1-4436-a423-8d10aeee6365 admin admin] PATH_INFO: `/v3/projects/bb095b7b4a1d48f7b8c43d14f8a25680/users/8c60fbd05a6648a38f50971a1523b531/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:08.392529 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1254/2474] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:08 2022] POST /identity/v3/projects => generated 403 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:08.393827 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf8c20-7671-4caf-ac0c-1a6e37145edd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:08.399053 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7537e-ebe1-4436-a423-8d10aeee6365 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb095b7b4a1d48f7b8c43d14f8a25680, user_id=8c60fbd05a6648a38f50971a1523b531, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:08.402586 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7537e-ebe1-4436-a423-8d10aeee6365 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:08.416994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1221/2475] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:08 2022] PUT /identity/v3/projects/bb095b7b4a1d48f7b8c43d14f8a25680/users/8c60fbd05a6648a38f50971a1523b531/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:08.421029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7537e-ebe1-4436-a423-8d10aeee6365 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:08.459397 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf8c20-7671-4caf-ac0c-1a6e37145edd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:08.481697 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7537e-ebe1-4436-a423-8d10aeee6365 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:08.486364 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797378a-ee21-4146-8855-e53f3f684f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:08.487035 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797378a-ee21-4146-8855-e53f3f684f3e admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:08.487313 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797378a-ee21-4146-8855-e53f3f684f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:08.487591 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797378a-ee21-4146-8855-e53f3f684f3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:08.488363 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797378a-ee21-4146-8855-e53f3f684f3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:08.489986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797378a-ee21-4146-8855-e53f3f684f3e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:08.500134 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1222/2476] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:08.504569 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797378a-ee21-4146-8855-e53f3f684f3e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:08.514595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797378a-ee21-4146-8855-e53f3f684f3e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:08.518060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193de77b-fc10-4e90-9b9f-e35670896eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:08.518763 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193de77b-fc10-4e90-9b9f-e35670896eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:08.519061 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193de77b-fc10-4e90-9b9f-e35670896eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:08.519163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac7a0a-155c-45fb-80e4-2c13a2fcc84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:08.519334 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193de77b-fc10-4e90-9b9f-e35670896eb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:08.520015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac7a0a-155c-45fb-80e4-2c13a2fcc84b admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:08.520118 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193de77b-fc10-4e90-9b9f-e35670896eb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:08.520295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac7a0a-155c-45fb-80e4-2c13a2fcc84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:08.520565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac7a0a-155c-45fb-80e4-2c13a2fcc84b admin admin] PATH_INFO: `/v3/projects/bb095b7b4a1d48f7b8c43d14f8a25680/users/8c60fbd05a6648a38f50971a1523b531/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:08.522027 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193de77b-fc10-4e90-9b9f-e35670896eb0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:08.522537 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac7a0a-155c-45fb-80e4-2c13a2fcc84b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb095b7b4a1d48f7b8c43d14f8a25680, user_id=8c60fbd05a6648a38f50971a1523b531, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:08.525183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac7a0a-155c-45fb-80e4-2c13a2fcc84b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:08.528465 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:08.531439 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:08.538663 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1223/2477] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:08 2022] PUT /identity/v3/projects/bb095b7b4a1d48f7b8c43d14f8a25680/users/8c60fbd05a6648a38f50971a1523b531/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:08.544140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1255/2478] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:08 2022] POST /identity/v3/users => generated 308 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:08.548216 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac7a0a-155c-45fb-80e4-2c13a2fcc84b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:08.548593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354b138-499a-4583-b71a-6f401c4f01f3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:08.548841 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354b138-499a-4583-b71a-6f401c4f01f3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:08.549226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354b138-499a-4583-b71a-6f401c4f01f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:08.575432 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f354b138-499a-4583-b71a-6f401c4f01f3 None None] MFA Rules not processed for user `8c60fbd05a6648a38f50971a1523b531`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:08.577093 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f354b138-499a-4583-b71a-6f401c4f01f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:08.616102 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac7a0a-155c-45fb-80e4-2c13a2fcc84b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:08.621542 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e2045-9eec-4ad9-beb6-2445b5bb4caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:08.622212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e2045-9eec-4ad9-beb6-2445b5bb4caa admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:08.622500 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e2045-9eec-4ad9-beb6-2445b5bb4caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:08.622779 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e2045-9eec-4ad9-beb6-2445b5bb4caa admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:08.623528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e2045-9eec-4ad9-beb6-2445b5bb4caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:08.625208 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e2045-9eec-4ad9-beb6-2445b5bb4caa admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:08.636355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1224/2479] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:08.640619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e2045-9eec-4ad9-beb6-2445b5bb4caa admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:08.645724 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1256/2480] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:08.652090 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e2045-9eec-4ad9-beb6-2445b5bb4caa admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:08.656616 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354b138-499a-4583-b71a-6f401c4f01f3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:08.657543 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2d977-9acc-48ef-a65b-54ad7f3d7113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:08.657934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f354b138-499a-4583-b71a-6f401c4f01f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:08.658430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2d977-9acc-48ef-a65b-54ad7f3d7113 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:08.658710 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2d977-9acc-48ef-a65b-54ad7f3d7113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:08.659015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2d977-9acc-48ef-a65b-54ad7f3d7113 admin admin] PATH_INFO: `/v3/projects/54f0f026c13540f8944d72ddc364a1f7/users/b2a5fa04d99f4a5f9a95376768fd0da9/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:08.674659 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2d977-9acc-48ef-a65b-54ad7f3d7113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54f0f026c13540f8944d72ddc364a1f7, user_id=b2a5fa04d99f4a5f9a95376768fd0da9, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:08.677366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2d977-9acc-48ef-a65b-54ad7f3d7113 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:08.690863 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1225/2481] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:08 2022] PUT /identity/v3/projects/54f0f026c13540f8944d72ddc364a1f7/users/b2a5fa04d99f4a5f9a95376768fd0da9/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:08.694823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2d977-9acc-48ef-a65b-54ad7f3d7113 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:08.715481 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354b138-499a-4583-b71a-6f401c4f01f3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:08.758603 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2d977-9acc-48ef-a65b-54ad7f3d7113 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:08.764066 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c520a1-7a4d-4d9e-aae2-0453e786423f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:08.764727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c520a1-7a4d-4d9e-aae2-0453e786423f admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:08.765012 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c520a1-7a4d-4d9e-aae2-0453e786423f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:08.765291 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c520a1-7a4d-4d9e-aae2-0453e786423f admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:08.765543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250cc9b-cf71-4af9-8b53-09882789ddda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:08.766017 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c520a1-7a4d-4d9e-aae2-0453e786423f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:08.766210 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250cc9b-cf71-4af9-8b53-09882789ddda service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:08.766501 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250cc9b-cf71-4af9-8b53-09882789ddda service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:08.766775 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250cc9b-cf71-4af9-8b53-09882789ddda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:08.767808 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c520a1-7a4d-4d9e-aae2-0453e786423f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:08.777967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1226/2482] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:23:08.782142 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c520a1-7a4d-4d9e-aae2-0453e786423f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:08.791942 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c520a1-7a4d-4d9e-aae2-0453e786423f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:08.797810 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8bf0c6-a930-474b-9452-e7fc981abfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:08.798684 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bf0c6-a930-474b-9452-e7fc981abfa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:08.798979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bf0c6-a930-474b-9452-e7fc981abfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:08.799256 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bf0c6-a930-474b-9452-e7fc981abfa9 admin admin] PATH_INFO: `/v3/projects/54f0f026c13540f8944d72ddc364a1f7/users/b2a5fa04d99f4a5f9a95376768fd0da9/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:08.801322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8bf0c6-a930-474b-9452-e7fc981abfa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54f0f026c13540f8944d72ddc364a1f7, user_id=b2a5fa04d99f4a5f9a95376768fd0da9, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:08.804024 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8bf0c6-a930-474b-9452-e7fc981abfa9 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:08.815157 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1227/2483] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:08 2022] PUT /identity/v3/projects/54f0f026c13540f8944d72ddc364a1f7/users/b2a5fa04d99f4a5f9a95376768fd0da9/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:08.817982 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:08.834767 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1250cc9b-cf71-4af9-8b53-09882789ddda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:08.836454 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1250cc9b-cf71-4af9-8b53-09882789ddda service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:08.847234 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1257/2484] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:23:08.876238 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8bf0c6-a930-474b-9452-e7fc981abfa9 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:08.933069 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:08.945096 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8bf0c6-a930-474b-9452-e7fc981abfa9 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:08.949729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001b96d-0e5b-4d1a-a9ec-c503d0f4360c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:08.950620 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001b96d-0e5b-4d1a-a9ec-c503d0f4360c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:08.950893 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001b96d-0e5b-4d1a-a9ec-c503d0f4360c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:08.951161 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001b96d-0e5b-4d1a-a9ec-c503d0f4360c admin admin] PATH_INFO: `/v3/users/4528c3346c414dc3bfe9c1638aa83f75` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:08.952391 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001b96d-0e5b-4d1a-a9ec-c503d0f4360c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4528c3346c414dc3bfe9c1638aa83f75)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:08.954317 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001b96d-0e5b-4d1a-a9ec-c503d0f4360c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:09.009697 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4001b96d-0e5b-4d1a-a9ec-c503d0f4360c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4528c3346c414dc3bfe9c1638aa83f75'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:09.010673 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4001b96d-0e5b-4d1a-a9ec-c503d0f4360c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4528c3346c414dc3bfe9c1638aa83f75'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:09.011712 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4001b96d-0e5b-4d1a-a9ec-c503d0f4360c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4528c3346c414dc3bfe9c1638aa83f75'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:09.025152 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4001b96d-0e5b-4d1a-a9ec-c503d0f4360c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4528c3346c414dc3bfe9c1638aa83f75'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:09.032170 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4001b96d-0e5b-4d1a-a9ec-c503d0f4360c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4528c3346c414dc3bfe9c1638aa83f75'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:09.046519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1228/2485] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:08 2022] DELETE /identity/v3/users/4528c3346c414dc3bfe9c1638aa83f75 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:09.051069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57dd2bd-6f6b-4af0-ac6b-105dc75077a4 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:09.051361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57dd2bd-6f6b-4af0-ac6b-105dc75077a4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:09.051676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57dd2bd-6f6b-4af0-ac6b-105dc75077a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:09.069112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c57dd2bd-6f6b-4af0-ac6b-105dc75077a4 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:09.070326 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c57dd2bd-6f6b-4af0-ac6b-105dc75077a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:09.142088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1258/2486] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:09.152055 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001b96d-0e5b-4d1a-a9ec-c503d0f4360c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:09.153571 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4001b96d-0e5b-4d1a-a9ec-c503d0f4360c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:09.222643 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001b96d-0e5b-4d1a-a9ec-c503d0f4360c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:09.226855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbd7ce-3d29-4f49-95fd-a9be841b7815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:09.227913 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbd7ce-3d29-4f49-95fd-a9be841b7815 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:09.227913 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbd7ce-3d29-4f49-95fd-a9be841b7815 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:09.228514 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbd7ce-3d29-4f49-95fd-a9be841b7815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:09.241402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbd7ce-3d29-4f49-95fd-a9be841b7815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:09.242991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbd7ce-3d29-4f49-95fd-a9be841b7815 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:09.254310 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1229/2487] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:23:09.546636 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c16ba-34ad-41db-8543-b4aba5021237 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:09.546901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c16ba-34ad-41db-8543-b4aba5021237 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:09.547219 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c16ba-34ad-41db-8543-b4aba5021237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:09.576054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbd7ce-3d29-4f49-95fd-a9be841b7815 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:09.577427 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-60dbd7ce-3d29-4f49-95fd-a9be841b7815 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:09.581293 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e40c16ba-34ad-41db-8543-b4aba5021237 None None] MFA Rules not processed for user `b2a5fa04d99f4a5f9a95376768fd0da9`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:09.582477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e40c16ba-34ad-41db-8543-b4aba5021237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:09.594561 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbd7ce-3d29-4f49-95fd-a9be841b7815 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:09.599949 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba9706-3c19-4e17-90a6-da1f564b7d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:09.600829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba9706-3c19-4e17-90a6-da1f564b7d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:09.601117 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba9706-3c19-4e17-90a6-da1f564b7d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:09.601630 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba9706-3c19-4e17-90a6-da1f564b7d0c admin admin] PATH_INFO: `/v3/projects/d9dbb9c7a0e74808b1513ee1702e4279` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:09.602921 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba9706-3c19-4e17-90a6-da1f564b7d0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9dbb9c7a0e74808b1513ee1702e4279)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:09.605519 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba9706-3c19-4e17-90a6-da1f564b7d0c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:09.662395 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4aba9706-3c19-4e17-90a6-da1f564b7d0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9dbb9c7a0e74808b1513ee1702e4279', 'request_id': 'req-4aba9706-3c19-4e17-90a6-da1f564b7d0c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:09.671211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4aba9706-3c19-4e17-90a6-da1f564b7d0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9dbb9c7a0e74808b1513ee1702e4279', 'request_id': 'req-4aba9706-3c19-4e17-90a6-da1f564b7d0c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:09.671409 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1259/2488] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:09 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:09.678206 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4aba9706-3c19-4e17-90a6-da1f564b7d0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9dbb9c7a0e74808b1513ee1702e4279', 'request_id': 'req-4aba9706-3c19-4e17-90a6-da1f564b7d0c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:09.680221 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:09.691883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4aba9706-3c19-4e17-90a6-da1f564b7d0c admin admin] The token cache is being invalidate because project d9dbb9c7a0e74808b1513ee1702e4279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:09.692123 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4aba9706-3c19-4e17-90a6-da1f564b7d0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:09.692917 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4aba9706-3c19-4e17-90a6-da1f564b7d0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:09.695090 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1230/2489] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:09 2022] DELETE /identity/v3/projects/d9dbb9c7a0e74808b1513ee1702e4279 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:09.700072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c16ba-34ad-41db-8543-b4aba5021237 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:09.711314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c16ba-34ad-41db-8543-b4aba5021237 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:09.777869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10d9a3-9cba-4e45-a89a-22e1b96d3826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:09.778786 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10d9a3-9cba-4e45-a89a-22e1b96d3826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:09.779072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10d9a3-9cba-4e45-a89a-22e1b96d3826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:09.779352 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10d9a3-9cba-4e45-a89a-22e1b96d3826 admin admin] PATH_INFO: `/v3/users/8c60fbd05a6648a38f50971a1523b531` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:09.780617 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10d9a3-9cba-4e45-a89a-22e1b96d3826 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c60fbd05a6648a38f50971a1523b531)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:09.782667 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10d9a3-9cba-4e45-a89a-22e1b96d3826 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:09.796205 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:09.831739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9f10d9a3-9cba-4e45-a89a-22e1b96d3826 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c60fbd05a6648a38f50971a1523b531'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:09.832692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9f10d9a3-9cba-4e45-a89a-22e1b96d3826 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c60fbd05a6648a38f50971a1523b531'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:09.833622 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9f10d9a3-9cba-4e45-a89a-22e1b96d3826 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c60fbd05a6648a38f50971a1523b531'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:09.846749 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9f10d9a3-9cba-4e45-a89a-22e1b96d3826 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c60fbd05a6648a38f50971a1523b531'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:09.854045 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9f10d9a3-9cba-4e45-a89a-22e1b96d3826 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c60fbd05a6648a38f50971a1523b531'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:09.868298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1260/2490] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:09 2022] DELETE /identity/v3/users/8c60fbd05a6648a38f50971a1523b531 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:10.305351 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba9706-3c19-4e17-90a6-da1f564b7d0c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:10.307051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4aba9706-3c19-4e17-90a6-da1f564b7d0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:10.321683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba9706-3c19-4e17-90a6-da1f564b7d0c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:10.384212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc24555-1781-4cdc-8b13-afc6e04c6e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:10.385099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc24555-1781-4cdc-8b13-afc6e04c6e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:10.385381 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc24555-1781-4cdc-8b13-afc6e04c6e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:10.385657 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc24555-1781-4cdc-8b13-afc6e04c6e29 admin admin] PATH_INFO: `/v3/projects/bb095b7b4a1d48f7b8c43d14f8a25680` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:10.386853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc24555-1781-4cdc-8b13-afc6e04c6e29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb095b7b4a1d48f7b8c43d14f8a25680)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:10.388823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc24555-1781-4cdc-8b13-afc6e04c6e29 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:10.447541 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5bc24555-1781-4cdc-8b13-afc6e04c6e29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb095b7b4a1d48f7b8c43d14f8a25680', 'request_id': 'req-5bc24555-1781-4cdc-8b13-afc6e04c6e29'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:10.459182 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5bc24555-1781-4cdc-8b13-afc6e04c6e29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb095b7b4a1d48f7b8c43d14f8a25680', 'request_id': 'req-5bc24555-1781-4cdc-8b13-afc6e04c6e29'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:10.466866 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5bc24555-1781-4cdc-8b13-afc6e04c6e29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb095b7b4a1d48f7b8c43d14f8a25680', 'request_id': 'req-5bc24555-1781-4cdc-8b13-afc6e04c6e29'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:10.481649 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5bc24555-1781-4cdc-8b13-afc6e04c6e29 admin admin] The token cache is being invalidate because project bb095b7b4a1d48f7b8c43d14f8a25680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:10.482034 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5bc24555-1781-4cdc-8b13-afc6e04c6e29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:10.482810 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5bc24555-1781-4cdc-8b13-afc6e04c6e29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:10.484693 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1231/2491] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:10 2022] DELETE /identity/v3/projects/bb095b7b4a1d48f7b8c43d14f8a25680 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:10.489813 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7715d-298a-41f1-8283-3e1bb38ac76c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:10.490095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7715d-298a-41f1-8283-3e1bb38ac76c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:10.490373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7715d-298a-41f1-8283-3e1bb38ac76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:10.510472 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7eb7715d-298a-41f1-8283-3e1bb38ac76c None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:10.510577 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7eb7715d-298a-41f1-8283-3e1bb38ac76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:10.573460 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1261/2492] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:10.582901 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b98e7d-e12f-4629-8ef6-d46de8e50097 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:10.583261 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b98e7d-e12f-4629-8ef6-d46de8e50097 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:10.583623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b98e7d-e12f-4629-8ef6-d46de8e50097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:10.603724 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-c8b98e7d-e12f-4629-8ef6-d46de8e50097 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:10.605183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c8b98e7d-e12f-4629-8ef6-d46de8e50097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:10.614262 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1232/2493] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:10.618624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7715d-298a-41f1-8283-3e1bb38ac76c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:10.628555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7715d-298a-41f1-8283-3e1bb38ac76c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:10.633617 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f7901-01eb-4d55-acc6-ddc93ff97f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:10.634352 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f7901-01eb-4d55-acc6-ddc93ff97f73 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:10.634700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f7901-01eb-4d55-acc6-ddc93ff97f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:10.635039 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f7901-01eb-4d55-acc6-ddc93ff97f73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:10.635988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f7901-01eb-4d55-acc6-ddc93ff97f73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:23:10.636300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f7901-01eb-4d55-acc6-ddc93ff97f73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:10.637914 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f7901-01eb-4d55-acc6-ddc93ff97f73 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:10.649881 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1262/2494] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:23:10.654051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b98e7d-e12f-4629-8ef6-d46de8e50097 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:10.663578 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b98e7d-e12f-4629-8ef6-d46de8e50097 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:10.668707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e48f4c4-237f-4a68-b813-6064db0bcc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:10.669453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48f4c4-237f-4a68-b813-6064db0bcc3a admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:10.669799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48f4c4-237f-4a68-b813-6064db0bcc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:10.670133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48f4c4-237f-4a68-b813-6064db0bcc3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:10.670896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e48f4c4-237f-4a68-b813-6064db0bcc3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:10.672903 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e48f4c4-237f-4a68-b813-6064db0bcc3a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:10.699309 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1233/2495] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:10 2022] POST /identity/v3/projects => generated 393 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:10.703492 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f7901-01eb-4d55-acc6-ddc93ff97f73 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:10.713471 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f7901-01eb-4d55-acc6-ddc93ff97f73 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:10.744562 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e48f4c4-237f-4a68-b813-6064db0bcc3a admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:10.746960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5e48f4c4-237f-4a68-b813-6064db0bcc3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:10.774534 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0567fb6-9fa4-4d29-bc89-2a644632be6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:10.775487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0567fb6-9fa4-4d29-bc89-2a644632be6a admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:10.775991 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0567fb6-9fa4-4d29-bc89-2a644632be6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:10.776272 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0567fb6-9fa4-4d29-bc89-2a644632be6a admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:10.777050 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0567fb6-9fa4-4d29-bc89-2a644632be6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:10.778983 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0567fb6-9fa4-4d29-bc89-2a644632be6a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:10.803883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1263/2496] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:10 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:10.805785 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0567fb6-9fa4-4d29-bc89-2a644632be6a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:10.808043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f0567fb6-9fa4-4d29-bc89-2a644632be6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:10.820098 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:10.821316 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0567fb6-9fa4-4d29-bc89-2a644632be6a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:10.826611 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82322cc9-1f4f-4e75-936a-17e7bdde5464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:10.827440 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82322cc9-1f4f-4e75-936a-17e7bdde5464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:10.827757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82322cc9-1f4f-4e75-936a-17e7bdde5464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:10.828033 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82322cc9-1f4f-4e75-936a-17e7bdde5464 admin admin] PATH_INFO: `/v3/users/cfd454e5ea9b415e9042199fa64db2d4` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:10.829215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82322cc9-1f4f-4e75-936a-17e7bdde5464 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfd454e5ea9b415e9042199fa64db2d4)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:10.831224 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82322cc9-1f4f-4e75-936a-17e7bdde5464 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:10.832983 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:10.838285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e48f4c4-237f-4a68-b813-6064db0bcc3a admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:10.846111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e24271-6328-4a61-b909-c2dded382ba3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5fa9b0b0314ee9bb7e16b8464307b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:10.846800 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e24271-6328-4a61-b909-c2dded382ba3 service placement] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:10.847088 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e24271-6328-4a61-b909-c2dded382ba3 service placement] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:10.847370 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e24271-6328-4a61-b909-c2dded382ba3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:10.849467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-03e24271-6328-4a61-b909-c2dded382ba3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:10.861474 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e24271-6328-4a61-b909-c2dded382ba3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:10.863108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e24271-6328-4a61-b909-c2dded382ba3 service placement] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:10.874000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1234/2497] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:10.874785 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e24271-6328-4a61-b909-c2dded382ba3 service placement] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:10.885263 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e24271-6328-4a61-b909-c2dded382ba3 service placement] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:10.887308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-82322cc9-1f4f-4e75-936a-17e7bdde5464 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfd454e5ea9b415e9042199fa64db2d4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:10.888080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-82322cc9-1f4f-4e75-936a-17e7bdde5464 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfd454e5ea9b415e9042199fa64db2d4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:10.888784 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-82322cc9-1f4f-4e75-936a-17e7bdde5464 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfd454e5ea9b415e9042199fa64db2d4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:10.891582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca7b42-8489-42fd-9f14-8b0166720e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:10.892789 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca7b42-8489-42fd-9f14-8b0166720e46 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:10.893074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca7b42-8489-42fd-9f14-8b0166720e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:10.893616 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca7b42-8489-42fd-9f14-8b0166720e46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:10.894339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca7b42-8489-42fd-9f14-8b0166720e46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:10.895993 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca7b42-8489-42fd-9f14-8b0166720e46 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:10.904849 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-82322cc9-1f4f-4e75-936a-17e7bdde5464 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfd454e5ea9b415e9042199fa64db2d4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:10.907081 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1235/2498] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:10.910772 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca7b42-8489-42fd-9f14-8b0166720e46 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:10.912522 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-46ca7b42-8489-42fd-9f14-8b0166720e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:10.912828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-82322cc9-1f4f-4e75-936a-17e7bdde5464 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfd454e5ea9b415e9042199fa64db2d4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:10.925382 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca7b42-8489-42fd-9f14-8b0166720e46 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:10.926581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1264/2499] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:10 2022] DELETE /identity/v3/users/cfd454e5ea9b415e9042199fa64db2d4 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:10.982134 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67f62c1-c8a0-4345-bbdf-55b38b8d2c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:10.982815 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f62c1-c8a0-4345-bbdf-55b38b8d2c9d admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:10.983097 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f62c1-c8a0-4345-bbdf-55b38b8d2c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:10.983368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f62c1-c8a0-4345-bbdf-55b38b8d2c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:10.984135 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67f62c1-c8a0-4345-bbdf-55b38b8d2c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:10.985781 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67f62c1-c8a0-4345-bbdf-55b38b8d2c9d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:11.000278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1236/2500] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:23:11.001310 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82322cc9-1f4f-4e75-936a-17e7bdde5464 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:11.012482 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82322cc9-1f4f-4e75-936a-17e7bdde5464 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:11.017582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa0167-8645-4e2c-9fdf-2e2a789848fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:11.018483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa0167-8645-4e2c-9fdf-2e2a789848fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:11.018759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa0167-8645-4e2c-9fdf-2e2a789848fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:11.019030 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa0167-8645-4e2c-9fdf-2e2a789848fd admin admin] PATH_INFO: `/v3/projects/93711c84abb7447aa9951d7bdb78df57/users/239aab5533be4cbfa995c0015cb4af5c/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:11.034993 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fa0167-8645-4e2c-9fdf-2e2a789848fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=93711c84abb7447aa9951d7bdb78df57, user_id=239aab5533be4cbfa995c0015cb4af5c, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:11.037692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fa0167-8645-4e2c-9fdf-2e2a789848fd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:11.051328 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1265/2501] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:11 2022] PUT /identity/v3/projects/93711c84abb7447aa9951d7bdb78df57/users/239aab5533be4cbfa995c0015cb4af5c/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:11.061475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac127bda-da98-4ac0-809a-68b916f70c1a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:11.061832 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac127bda-da98-4ac0-809a-68b916f70c1a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:11.062118 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac127bda-da98-4ac0-809a-68b916f70c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:11.092719 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ac127bda-da98-4ac0-809a-68b916f70c1a None None] MFA Rules not processed for user `239aab5533be4cbfa995c0015cb4af5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:11.093999 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ac127bda-da98-4ac0-809a-68b916f70c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:11.107650 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa0167-8645-4e2c-9fdf-2e2a789848fd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:11.109440 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-21fa0167-8645-4e2c-9fdf-2e2a789848fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:11.168769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1237/2502] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:11.173111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac127bda-da98-4ac0-809a-68b916f70c1a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:11.186908 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa0167-8645-4e2c-9fdf-2e2a789848fd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:11.191583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb399c-a559-4828-80ca-ba3a252b41fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:11.192484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb399c-a559-4828-80ca-ba3a252b41fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:11.192760 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb399c-a559-4828-80ca-ba3a252b41fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:11.193030 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb399c-a559-4828-80ca-ba3a252b41fc admin admin] PATH_INFO: `/v3/projects/828cf440c5dc4d7ab5da4b72b8846eea` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:11.194202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb399c-a559-4828-80ca-ba3a252b41fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=828cf440c5dc4d7ab5da4b72b8846eea)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:11.196167 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb399c-a559-4828-80ca-ba3a252b41fc admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:11.233877 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac127bda-da98-4ac0-809a-68b916f70c1a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:11.239804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:11.246070 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-60bb399c-a559-4828-80ca-ba3a252b41fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '828cf440c5dc4d7ab5da4b72b8846eea', 'request_id': 'req-60bb399c-a559-4828-80ca-ba3a252b41fc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:11.255152 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-60bb399c-a559-4828-80ca-ba3a252b41fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '828cf440c5dc4d7ab5da4b72b8846eea', 'request_id': 'req-60bb399c-a559-4828-80ca-ba3a252b41fc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:11.256542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:11.264287 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-60bb399c-a559-4828-80ca-ba3a252b41fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '828cf440c5dc4d7ab5da4b72b8846eea', 'request_id': 'req-60bb399c-a559-4828-80ca-ba3a252b41fc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:11.275853 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-60bb399c-a559-4828-80ca-ba3a252b41fc admin admin] The token cache is being invalidate because project 828cf440c5dc4d7ab5da4b72b8846eea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:11.276151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-60bb399c-a559-4828-80ca-ba3a252b41fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:11.276830 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-60bb399c-a559-4828-80ca-ba3a252b41fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:11.278576 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1266/2503] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:11 2022] DELETE /identity/v3/projects/828cf440c5dc4d7ab5da4b72b8846eea => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:11.282521 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb399c-a559-4828-80ca-ba3a252b41fc admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:11.283998 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-60bb399c-a559-4828-80ca-ba3a252b41fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:11.298954 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1d27e-b512-4db1-a53c-c40497c7b991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:11.299700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1d27e-b512-4db1-a53c-c40497c7b991 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:11.299938 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1d27e-b512-4db1-a53c-c40497c7b991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:11.300212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1d27e-b512-4db1-a53c-c40497c7b991 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:11.300901 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1d27e-b512-4db1-a53c-c40497c7b991 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:11.302819 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1d27e-b512-4db1-a53c-c40497c7b991 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:11.332101 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1238/2504] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:11 2022] POST /identity/v3/projects => generated 391 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:11.333136 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1d27e-b512-4db1-a53c-c40497c7b991 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:11.334767 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7ac1d27e-b512-4db1-a53c-c40497c7b991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:11.352890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb399c-a559-4828-80ca-ba3a252b41fc admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:11.401273 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1d27e-b512-4db1-a53c-c40497c7b991 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:11.405936 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe8f32-5fff-4656-b9fb-9a554efa5592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:11.406609 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe8f32-5fff-4656-b9fb-9a554efa5592 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:11.406888 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe8f32-5fff-4656-b9fb-9a554efa5592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:11.407163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe8f32-5fff-4656-b9fb-9a554efa5592 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:11.407881 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe8f32-5fff-4656-b9fb-9a554efa5592 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:11.409807 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe8f32-5fff-4656-b9fb-9a554efa5592 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:11.412114 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b1211-ba67-47d1-b0f1-c0156b2121c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:11.413004 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b1211-ba67-47d1-b0f1-c0156b2121c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:11.413267 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b1211-ba67-47d1-b0f1-c0156b2121c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:11.413544 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b1211-ba67-47d1-b0f1-c0156b2121c2 admin admin] PATH_INFO: `/v3/users/b2a5fa04d99f4a5f9a95376768fd0da9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:11.414717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b1211-ba67-47d1-b0f1-c0156b2121c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2a5fa04d99f4a5f9a95376768fd0da9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:11.416697 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b1211-ba67-47d1-b0f1-c0156b2121c2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:11.429927 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1239/2505] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:11 2022] POST /identity/v3/users => generated 302 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:11.433600 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe8f32-5fff-4656-b9fb-9a554efa5592 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:11.443825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe8f32-5fff-4656-b9fb-9a554efa5592 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:11.449344 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb925ca-207f-4672-8318-14ff2fe36a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:11.449995 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb925ca-207f-4672-8318-14ff2fe36a5d admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:11.450435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb925ca-207f-4672-8318-14ff2fe36a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:11.450703 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb925ca-207f-4672-8318-14ff2fe36a5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:11.451466 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb925ca-207f-4672-8318-14ff2fe36a5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:11.453128 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb925ca-207f-4672-8318-14ff2fe36a5d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:11.463266 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1240/2506] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:11.467314 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb925ca-207f-4672-8318-14ff2fe36a5d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:11.467721 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-644b1211-ba67-47d1-b0f1-c0156b2121c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2a5fa04d99f4a5f9a95376768fd0da9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:11.468395 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-644b1211-ba67-47d1-b0f1-c0156b2121c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2a5fa04d99f4a5f9a95376768fd0da9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:11.469007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-644b1211-ba67-47d1-b0f1-c0156b2121c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2a5fa04d99f4a5f9a95376768fd0da9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:11.477121 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb925ca-207f-4672-8318-14ff2fe36a5d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:11.481781 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659df1e-7520-4705-81b4-65faa07709df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:11.482208 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-644b1211-ba67-47d1-b0f1-c0156b2121c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2a5fa04d99f4a5f9a95376768fd0da9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:11.482633 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659df1e-7520-4705-81b4-65faa07709df admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:11.482912 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659df1e-7520-4705-81b4-65faa07709df admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:11.483179 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659df1e-7520-4705-81b4-65faa07709df admin admin] PATH_INFO: `/v3/projects/5ec3bb78af3e4868afef2863a5568ba5/users/4053a42eb22e42c6905c31041ab7e5fa/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:11.489375 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-644b1211-ba67-47d1-b0f1-c0156b2121c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2a5fa04d99f4a5f9a95376768fd0da9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:11.498780 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659df1e-7520-4705-81b4-65faa07709df admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ec3bb78af3e4868afef2863a5568ba5, user_id=4053a42eb22e42c6905c31041ab7e5fa, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:11.501459 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659df1e-7520-4705-81b4-65faa07709df admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:11.503321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1267/2507] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:11 2022] DELETE /identity/v3/users/b2a5fa04d99f4a5f9a95376768fd0da9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:11.516657 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1241/2508] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:11 2022] PUT /identity/v3/projects/5ec3bb78af3e4868afef2863a5568ba5/users/4053a42eb22e42c6905c31041ab7e5fa/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:11.520579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b1211-ba67-47d1-b0f1-c0156b2121c2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:11.522106 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-644b1211-ba67-47d1-b0f1-c0156b2121c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:11.536292 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:11.589359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b1211-ba67-47d1-b0f1-c0156b2121c2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:11.644841 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc8268-ca0a-42e4-a3cb-3e523d87bac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:11.645522 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc8268-ca0a-42e4-a3cb-3e523d87bac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:11.645806 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc8268-ca0a-42e4-a3cb-3e523d87bac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:11.646088 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc8268-ca0a-42e4-a3cb-3e523d87bac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:11.646819 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc8268-ca0a-42e4-a3cb-3e523d87bac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:11.648481 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc8268-ca0a-42e4-a3cb-3e523d87bac6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:11.658593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1268/2509] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:11.663096 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659df1e-7520-4705-81b4-65faa07709df admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:11.674976 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659df1e-7520-4705-81b4-65faa07709df admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:11.680341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e00c9-bf9e-4455-9b6e-91524da37eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:11.680560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc8268-ca0a-42e4-a3cb-3e523d87bac6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:11.681259 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e00c9-bf9e-4455-9b6e-91524da37eff admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:11.681556 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e00c9-bf9e-4455-9b6e-91524da37eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:11.681820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e00c9-bf9e-4455-9b6e-91524da37eff admin admin] PATH_INFO: `/v3/projects/5ec3bb78af3e4868afef2863a5568ba5/users/4053a42eb22e42c6905c31041ab7e5fa/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:11.682314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d6bc8268-ca0a-42e4-a3cb-3e523d87bac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:11.683976 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e00c9-bf9e-4455-9b6e-91524da37eff admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ec3bb78af3e4868afef2863a5568ba5, user_id=4053a42eb22e42c6905c31041ab7e5fa, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:11.686692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e00c9-bf9e-4455-9b6e-91524da37eff admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:11.696145 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc8268-ca0a-42e4-a3cb-3e523d87bac6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:11.699299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1242/2510] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:11 2022] PUT /identity/v3/projects/5ec3bb78af3e4868afef2863a5568ba5/users/4053a42eb22e42c6905c31041ab7e5fa/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:11.709813 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab073fda-83fa-4163-9c8d-dd5b3a669f1c None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:11.710148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab073fda-83fa-4163-9c8d-dd5b3a669f1c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:11.710480 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab073fda-83fa-4163-9c8d-dd5b3a669f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:11.738018 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ab073fda-83fa-4163-9c8d-dd5b3a669f1c None None] MFA Rules not processed for user `4053a42eb22e42c6905c31041ab7e5fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:11.739439 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ab073fda-83fa-4163-9c8d-dd5b3a669f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:11.761741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232695f0-befc-4e15-9fbf-d6aeb3229cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:11.762601 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232695f0-befc-4e15-9fbf-d6aeb3229cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:11.762884 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232695f0-befc-4e15-9fbf-d6aeb3229cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:11.763159 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232695f0-befc-4e15-9fbf-d6aeb3229cd8 admin admin] PATH_INFO: `/v3/projects/54f0f026c13540f8944d72ddc364a1f7` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:11.764383 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232695f0-befc-4e15-9fbf-d6aeb3229cd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54f0f026c13540f8944d72ddc364a1f7)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:11.765352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:11.766319 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232695f0-befc-4e15-9fbf-d6aeb3229cd8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:11.813662 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1243/2511] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:11 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:11.820569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-232695f0-befc-4e15-9fbf-d6aeb3229cd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54f0f026c13540f8944d72ddc364a1f7', 'request_id': 'req-232695f0-befc-4e15-9fbf-d6aeb3229cd8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:11.824853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab073fda-83fa-4163-9c8d-dd5b3a669f1c None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:11.826990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ab073fda-83fa-4163-9c8d-dd5b3a669f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:11.829901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-232695f0-befc-4e15-9fbf-d6aeb3229cd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54f0f026c13540f8944d72ddc364a1f7', 'request_id': 'req-232695f0-befc-4e15-9fbf-d6aeb3229cd8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:11.831274 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:11.838775 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-232695f0-befc-4e15-9fbf-d6aeb3229cd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54f0f026c13540f8944d72ddc364a1f7', 'request_id': 'req-232695f0-befc-4e15-9fbf-d6aeb3229cd8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:11.851866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-232695f0-befc-4e15-9fbf-d6aeb3229cd8 admin admin] The token cache is being invalidate because project 54f0f026c13540f8944d72ddc364a1f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:11.852158 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-232695f0-befc-4e15-9fbf-d6aeb3229cd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:11.852822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-232695f0-befc-4e15-9fbf-d6aeb3229cd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:11.854473 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1269/2512] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:11 2022] DELETE /identity/v3/projects/54f0f026c13540f8944d72ddc364a1f7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:11.859104 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20037e9d-8073-4d7f-8dc5-efa7ed23a6c2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:11.859391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20037e9d-8073-4d7f-8dc5-efa7ed23a6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:11.859697 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20037e9d-8073-4d7f-8dc5-efa7ed23a6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:11.877376 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-20037e9d-8073-4d7f-8dc5-efa7ed23a6c2 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:11.878587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-20037e9d-8073-4d7f-8dc5-efa7ed23a6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:11.889737 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab073fda-83fa-4163-9c8d-dd5b3a669f1c None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:11.937697 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3131436-8830-4228-be7a-e00deaa6649e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:11.937925 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1270/2513] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:11.939052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3131436-8830-4228-be7a-e00deaa6649e service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:11.939349 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3131436-8830-4228-be7a-e00deaa6649e service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:11.939704 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3131436-8830-4228-be7a-e00deaa6649e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:11.941569 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b3131436-8830-4228-be7a-e00deaa6649e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:11.945804 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5be6f-f6fa-45a5-a82e-42abaf1b5b5e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:11.946093 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5be6f-f6fa-45a5-a82e-42abaf1b5b5e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:11.946370 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5be6f-f6fa-45a5-a82e-42abaf1b5b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:11.964240 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-01e5be6f-f6fa-45a5-a82e-42abaf1b5b5e None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:11.965467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-01e5be6f-f6fa-45a5-a82e-42abaf1b5b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:11.973391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1271/2514] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:11.977611 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5be6f-f6fa-45a5-a82e-42abaf1b5b5e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:11.986743 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5be6f-f6fa-45a5-a82e-42abaf1b5b5e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:11.992411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52513206-b712-45f4-ad55-ae640748145a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:11.993155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52513206-b712-45f4-ad55-ae640748145a admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:11.993503 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52513206-b712-45f4-ad55-ae640748145a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:11.994078 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52513206-b712-45f4-ad55-ae640748145a admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:11.994573 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52513206-b712-45f4-ad55-ae640748145a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:23:11.994881 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52513206-b712-45f4-ad55-ae640748145a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:11.996691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52513206-b712-45f4-ad55-ae640748145a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:12.008753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1272/2515] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:23:12.009368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3131436-8830-4228-be7a-e00deaa6649e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:12.010964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3131436-8830-4228-be7a-e00deaa6649e service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:12.013225 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52513206-b712-45f4-ad55-ae640748145a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:12.020989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1244/2516] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:11 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:23:12.023233 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52513206-b712-45f4-ad55-ae640748145a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:12.028045 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f2f31-01f4-413b-ae73-b2fe87054c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:12.028641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f2f31-01f4-413b-ae73-b2fe87054c3d admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:12.028925 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f2f31-01f4-413b-ae73-b2fe87054c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:12.029355 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f2f31-01f4-413b-ae73-b2fe87054c3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:12.030539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f2f31-01f4-413b-ae73-b2fe87054c3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:12.032910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f2f31-01f4-413b-ae73-b2fe87054c3d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:12.069960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1273/2517] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:12 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:12.071126 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3131436-8830-4228-be7a-e00deaa6649e service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:12.074078 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f2f31-01f4-413b-ae73-b2fe87054c3d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:12.075855 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9b2f2f31-01f4-413b-ae73-b2fe87054c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:12.082806 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3131436-8830-4228-be7a-e00deaa6649e service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:12.107579 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:12.142605 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f2f31-01f4-413b-ae73-b2fe87054c3d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:12.143359 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35679c2-0173-46e8-8a02-b0d4894606ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:12.144072 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35679c2-0173-46e8-8a02-b0d4894606ad admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:12.144364 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35679c2-0173-46e8-8a02-b0d4894606ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:12.144629 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35679c2-0173-46e8-8a02-b0d4894606ad admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:12.145328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35679c2-0173-46e8-8a02-b0d4894606ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:12.146375 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae96258-54d8-498a-a3f8-054f1eaa1ef5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:12.147029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae96258-54d8-498a-a3f8-054f1eaa1ef5 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:12.147262 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35679c2-0173-46e8-8a02-b0d4894606ad admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:12.147379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae96258-54d8-498a-a3f8-054f1eaa1ef5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:12.147576 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae96258-54d8-498a-a3f8-054f1eaa1ef5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:12.149471 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bae96258-54d8-498a-a3f8-054f1eaa1ef5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:12.168143 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1245/2518] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:12 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:12.172437 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35679c2-0173-46e8-8a02-b0d4894606ad admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:12.183654 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35679c2-0173-46e8-8a02-b0d4894606ad admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:12.188850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee3da4-b54e-4d2f-9242-0f832d8e3e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:12.189521 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee3da4-b54e-4d2f-9242-0f832d8e3e1c admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:12.189805 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee3da4-b54e-4d2f-9242-0f832d8e3e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:12.190084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee3da4-b54e-4d2f-9242-0f832d8e3e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:12.190811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee3da4-b54e-4d2f-9242-0f832d8e3e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:12.192474 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee3da4-b54e-4d2f-9242-0f832d8e3e1c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:12.203777 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1246/2519] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:23:12.207672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee3da4-b54e-4d2f-9242-0f832d8e3e1c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:12.213402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae96258-54d8-498a-a3f8-054f1eaa1ef5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:12.214867 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae96258-54d8-498a-a3f8-054f1eaa1ef5 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:12.217931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee3da4-b54e-4d2f-9242-0f832d8e3e1c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:12.218474 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:12.223807 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869a709-bf39-4b40-8a03-bd4e61a75e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:12.224511 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869a709-bf39-4b40-8a03-bd4e61a75e8e admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:12.224787 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869a709-bf39-4b40-8a03-bd4e61a75e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:12.225025 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869a709-bf39-4b40-8a03-bd4e61a75e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:12.225863 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869a709-bf39-4b40-8a03-bd4e61a75e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:12.226575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1274/2520] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:23:12.227682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869a709-bf39-4b40-8a03-bd4e61a75e8e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:12.241840 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1247/2521] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:12.243121 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae96258-54d8-498a-a3f8-054f1eaa1ef5 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:12.254268 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae96258-54d8-498a-a3f8-054f1eaa1ef5 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:12.259945 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3fa41-a2b4-4146-9235-112e800f5579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:12.260909 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3fa41-a2b4-4146-9235-112e800f5579 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:12.261192 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3fa41-a2b4-4146-9235-112e800f5579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:12.261472 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3fa41-a2b4-4146-9235-112e800f5579 admin admin] PATH_INFO: `/v3/projects/fd1975df37404dce8dc32d67f3f18602/users/b6be15e4eb274ca3b679d31409193614/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:12.277381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3fa41-a2b4-4146-9235-112e800f5579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd1975df37404dce8dc32d67f3f18602, user_id=b6be15e4eb274ca3b679d31409193614, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:12.280108 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3fa41-a2b4-4146-9235-112e800f5579 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:12.293753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1275/2522] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:12 2022] PUT /identity/v3/projects/fd1975df37404dce8dc32d67f3f18602/users/b6be15e4eb274ca3b679d31409193614/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:12.304175 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78633239-41f0-4378-b3b9-623c59a55bf6 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:12.304465 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78633239-41f0-4378-b3b9-623c59a55bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:12.304739 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78633239-41f0-4378-b3b9-623c59a55bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:12.331946 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-78633239-41f0-4378-b3b9-623c59a55bf6 None None] MFA Rules not processed for user `b6be15e4eb274ca3b679d31409193614`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:12.333183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-78633239-41f0-4378-b3b9-623c59a55bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:12.406912 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1248/2523] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:12.411661 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3fa41-a2b4-4146-9235-112e800f5579 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:12.481234 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3fa41-a2b4-4146-9235-112e800f5579 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:12.495396 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1dec7-8d0f-4311-87da-328526e8d899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:12.495823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1dec7-8d0f-4311-87da-328526e8d899 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:12.496103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1dec7-8d0f-4311-87da-328526e8d899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:12.496382 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1dec7-8d0f-4311-87da-328526e8d899 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:12.497175 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1dec7-8d0f-4311-87da-328526e8d899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:12.499189 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1dec7-8d0f-4311-87da-328526e8d899 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:12.534317 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1276/2524] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:12 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:12.535462 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78633239-41f0-4378-b3b9-623c59a55bf6 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:12.546687 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78633239-41f0-4378-b3b9-623c59a55bf6 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:12.605540 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a426c2f-546b-4722-8327-16fa96cf0991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:12.606256 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a426c2f-546b-4722-8327-16fa96cf0991 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:12.606550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a426c2f-546b-4722-8327-16fa96cf0991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:12.606914 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a426c2f-546b-4722-8327-16fa96cf0991 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:12.607895 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a426c2f-546b-4722-8327-16fa96cf0991 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:12.610129 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a426c2f-546b-4722-8327-16fa96cf0991 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:12.632167 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1249/2525] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:12 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:12.636253 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1dec7-8d0f-4311-87da-328526e8d899 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:12.646778 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1dec7-8d0f-4311-87da-328526e8d899 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:12.652947 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a22de6-bb32-4b88-870a-1276303a7efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:12.653612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a22de6-bb32-4b88-870a-1276303a7efd admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:12.653891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a22de6-bb32-4b88-870a-1276303a7efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:12.654162 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a22de6-bb32-4b88-870a-1276303a7efd admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:12.654882 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a22de6-bb32-4b88-870a-1276303a7efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:12.656541 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a22de6-bb32-4b88-870a-1276303a7efd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:12.667832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1277/2526] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:12.672326 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a426c2f-546b-4722-8327-16fa96cf0991 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:12.683336 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a426c2f-546b-4722-8327-16fa96cf0991 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:12.689896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d16b8-1d84-4d02-bb76-dcf5228a2be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:12.690786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d16b8-1d84-4d02-bb76-dcf5228a2be8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:12.691069 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d16b8-1d84-4d02-bb76-dcf5228a2be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:12.691357 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d16b8-1d84-4d02-bb76-dcf5228a2be8 admin admin] PATH_INFO: `/v3/projects/5a8bcbc8381e4a769208732b1bfb31c2/users/78f666f887d24f8c8b403d57f40d4220/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:12.703786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:12.711143 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d16b8-1d84-4d02-bb76-dcf5228a2be8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a8bcbc8381e4a769208732b1bfb31c2, user_id=78f666f887d24f8c8b403d57f40d4220, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:12.714745 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d16b8-1d84-4d02-bb76-dcf5228a2be8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:12.729679 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1250/2527] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:12 2022] PUT /identity/v3/projects/5a8bcbc8381e4a769208732b1bfb31c2/users/78f666f887d24f8c8b403d57f40d4220/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:12.734219 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a22de6-bb32-4b88-870a-1276303a7efd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:12.745202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a22de6-bb32-4b88-870a-1276303a7efd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:12.813218 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790c5cc-d259-490e-8134-e9f2aea2b648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:12.814068 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790c5cc-d259-490e-8134-e9f2aea2b648 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:12.814217 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790c5cc-d259-490e-8134-e9f2aea2b648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:12.814439 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790c5cc-d259-490e-8134-e9f2aea2b648 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:12.815141 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790c5cc-d259-490e-8134-e9f2aea2b648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:12.817051 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790c5cc-d259-490e-8134-e9f2aea2b648 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:12.828244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1278/2528] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:12.832869 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d16b8-1d84-4d02-bb76-dcf5228a2be8 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:12.844689 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d16b8-1d84-4d02-bb76-dcf5228a2be8 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:12.850364 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d6e04-e61f-445e-9ca2-939a73a5bc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:12.851301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d6e04-e61f-445e-9ca2-939a73a5bc6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:12.851586 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d6e04-e61f-445e-9ca2-939a73a5bc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:12.851892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d6e04-e61f-445e-9ca2-939a73a5bc6c admin admin] PATH_INFO: `/v3/projects/5a8bcbc8381e4a769208732b1bfb31c2/users/78f666f887d24f8c8b403d57f40d4220/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:12.854022 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d6e04-e61f-445e-9ca2-939a73a5bc6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a8bcbc8381e4a769208732b1bfb31c2, user_id=78f666f887d24f8c8b403d57f40d4220, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:12.856970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d6e04-e61f-445e-9ca2-939a73a5bc6c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:12.869262 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1251/2529] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:12 2022] PUT /identity/v3/projects/5a8bcbc8381e4a769208732b1bfb31c2/users/78f666f887d24f8c8b403d57f40d4220/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:12.881245 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f94de4-bc8d-4fb7-ab70-7efdb5d673a1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:12.881532 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f94de4-bc8d-4fb7-ab70-7efdb5d673a1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:12.881805 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f94de4-bc8d-4fb7-ab70-7efdb5d673a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:12.911151 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-86f94de4-bc8d-4fb7-ab70-7efdb5d673a1 None None] MFA Rules not processed for user `78f666f887d24f8c8b403d57f40d4220`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:12.912427 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-86f94de4-bc8d-4fb7-ab70-7efdb5d673a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:12.955581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:12.996965 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1279/2530] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:12 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:13.012528 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d6e04-e61f-445e-9ca2-939a73a5bc6c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:13.084510 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d6e04-e61f-445e-9ca2-939a73a5bc6c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:13.089735 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eceec0-6c53-4c3c-a05b-0976a19d9634 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:13.090387 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eceec0-6c53-4c3c-a05b-0976a19d9634 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:13.090738 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eceec0-6c53-4c3c-a05b-0976a19d9634 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:13.091078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eceec0-6c53-4c3c-a05b-0976a19d9634 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:13.103387 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eceec0-6c53-4c3c-a05b-0976a19d9634 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:13.105338 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eceec0-6c53-4c3c-a05b-0976a19d9634 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:13.116855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1252/2531] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:13 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:13.516451 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f94de4-bc8d-4fb7-ab70-7efdb5d673a1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:13.580615 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f94de4-bc8d-4fb7-ab70-7efdb5d673a1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:13.585339 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458c1c2-a6e3-4421-b240-498c432a5bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:13.586221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458c1c2-a6e3-4421-b240-498c432a5bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:13.586496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458c1c2-a6e3-4421-b240-498c432a5bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:13.586806 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458c1c2-a6e3-4421-b240-498c432a5bfe admin admin] PATH_INFO: `/v3/users/b6be15e4eb274ca3b679d31409193614` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:13.588345 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c458c1c2-a6e3-4421-b240-498c432a5bfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6be15e4eb274ca3b679d31409193614)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:13.590781 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c458c1c2-a6e3-4421-b240-498c432a5bfe admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:13.642614 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c458c1c2-a6e3-4421-b240-498c432a5bfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6be15e4eb274ca3b679d31409193614'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:13.643267 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c458c1c2-a6e3-4421-b240-498c432a5bfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6be15e4eb274ca3b679d31409193614'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:13.643901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c458c1c2-a6e3-4421-b240-498c432a5bfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6be15e4eb274ca3b679d31409193614'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:13.656270 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c458c1c2-a6e3-4421-b240-498c432a5bfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6be15e4eb274ca3b679d31409193614'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:13.664639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c458c1c2-a6e3-4421-b240-498c432a5bfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6be15e4eb274ca3b679d31409193614'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:13.677168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1280/2532] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:13 2022] DELETE /identity/v3/users/b6be15e4eb274ca3b679d31409193614 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:13.681764 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0a522-4bbc-43cf-8a3b-437a5464613f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:13.682233 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0a522-4bbc-43cf-8a3b-437a5464613f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:13.682338 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0a522-4bbc-43cf-8a3b-437a5464613f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:13.700164 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-48d0a522-4bbc-43cf-8a3b-437a5464613f None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:13.701359 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-48d0a522-4bbc-43cf-8a3b-437a5464613f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:13.764260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1253/2533] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:13.774177 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458c1c2-a6e3-4421-b240-498c432a5bfe admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:13.775708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c458c1c2-a6e3-4421-b240-498c432a5bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:13.830743 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:13.835387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458c1c2-a6e3-4421-b240-498c432a5bfe admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:13.841389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff926dbb-3974-4fc3-b4dd-ebaaa31f01ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:13.842320 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff926dbb-3974-4fc3-b4dd-ebaaa31f01ef service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:13.842941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff926dbb-3974-4fc3-b4dd-ebaaa31f01ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:13.843158 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff926dbb-3974-4fc3-b4dd-ebaaa31f01ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:13.854060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff926dbb-3974-4fc3-b4dd-ebaaa31f01ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:13.855645 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff926dbb-3974-4fc3-b4dd-ebaaa31f01ef service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:13.866085 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1281/2534] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:23:14.144570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0a522-4bbc-43cf-8a3b-437a5464613f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:14.146173 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-48d0a522-4bbc-43cf-8a3b-437a5464613f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:14.160731 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0a522-4bbc-43cf-8a3b-437a5464613f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:14.168601 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad01ada-e461-49e3-a30c-40726202e291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:14.169395 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad01ada-e461-49e3-a30c-40726202e291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:14.169720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad01ada-e461-49e3-a30c-40726202e291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:14.169943 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad01ada-e461-49e3-a30c-40726202e291 admin admin] PATH_INFO: `/v3/projects/fd1975df37404dce8dc32d67f3f18602` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:14.171148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad01ada-e461-49e3-a30c-40726202e291 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd1975df37404dce8dc32d67f3f18602)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:14.173163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad01ada-e461-49e3-a30c-40726202e291 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:14.229547 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9ad01ada-e461-49e3-a30c-40726202e291 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd1975df37404dce8dc32d67f3f18602', 'request_id': 'req-9ad01ada-e461-49e3-a30c-40726202e291'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:14.237186 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9ad01ada-e461-49e3-a30c-40726202e291 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd1975df37404dce8dc32d67f3f18602', 'request_id': 'req-9ad01ada-e461-49e3-a30c-40726202e291'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:14.242888 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9ad01ada-e461-49e3-a30c-40726202e291 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd1975df37404dce8dc32d67f3f18602', 'request_id': 'req-9ad01ada-e461-49e3-a30c-40726202e291'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:14.254869 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9ad01ada-e461-49e3-a30c-40726202e291 admin admin] The token cache is being invalidate because project fd1975df37404dce8dc32d67f3f18602 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:14.255161 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9ad01ada-e461-49e3-a30c-40726202e291 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:14.255880 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9ad01ada-e461-49e3-a30c-40726202e291 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:14.257422 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1254/2535] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:14 2022] DELETE /identity/v3/projects/fd1975df37404dce8dc32d67f3f18602 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:14.261500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff926dbb-3974-4fc3-b4dd-ebaaa31f01ef service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:14.271322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff926dbb-3974-4fc3-b4dd-ebaaa31f01ef service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:14.329200 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f517625-9880-4309-8f97-cabc7ceea38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:14.330264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f517625-9880-4309-8f97-cabc7ceea38a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:14.330541 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f517625-9880-4309-8f97-cabc7ceea38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:14.330820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f517625-9880-4309-8f97-cabc7ceea38a admin admin] PATH_INFO: `/v3/users/78f666f887d24f8c8b403d57f40d4220` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:14.332307 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f517625-9880-4309-8f97-cabc7ceea38a admin admin] RBAC: Authorizing `identity:delete_user(user_id=78f666f887d24f8c8b403d57f40d4220)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:14.334283 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f517625-9880-4309-8f97-cabc7ceea38a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:14.386011 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1f517625-9880-4309-8f97-cabc7ceea38a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78f666f887d24f8c8b403d57f40d4220'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:14.386675 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1f517625-9880-4309-8f97-cabc7ceea38a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78f666f887d24f8c8b403d57f40d4220'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:14.387292 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1f517625-9880-4309-8f97-cabc7ceea38a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78f666f887d24f8c8b403d57f40d4220'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:14.399052 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1f517625-9880-4309-8f97-cabc7ceea38a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78f666f887d24f8c8b403d57f40d4220'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:14.405990 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1f517625-9880-4309-8f97-cabc7ceea38a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78f666f887d24f8c8b403d57f40d4220'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:14.420982 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1282/2536] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:14 2022] DELETE /identity/v3/users/78f666f887d24f8c8b403d57f40d4220 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:14.713400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad01ada-e461-49e3-a30c-40726202e291 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:14.714935 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9ad01ada-e461-49e3-a30c-40726202e291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:14.723449 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:14.730787 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad01ada-e461-49e3-a30c-40726202e291 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:14.791003 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38cdeb8-98b3-4454-9b76-c4af68562b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:14.791877 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cdeb8-98b3-4454-9b76-c4af68562b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:14.792148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cdeb8-98b3-4454-9b76-c4af68562b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:14.792416 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cdeb8-98b3-4454-9b76-c4af68562b34 admin admin] PATH_INFO: `/v3/projects/5a8bcbc8381e4a769208732b1bfb31c2` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:14.793570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38cdeb8-98b3-4454-9b76-c4af68562b34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a8bcbc8381e4a769208732b1bfb31c2)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:14.796016 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38cdeb8-98b3-4454-9b76-c4af68562b34 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:14.844289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c38cdeb8-98b3-4454-9b76-c4af68562b34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a8bcbc8381e4a769208732b1bfb31c2', 'request_id': 'req-c38cdeb8-98b3-4454-9b76-c4af68562b34'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:14.852661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c38cdeb8-98b3-4454-9b76-c4af68562b34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a8bcbc8381e4a769208732b1bfb31c2', 'request_id': 'req-c38cdeb8-98b3-4454-9b76-c4af68562b34'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:14.861341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c38cdeb8-98b3-4454-9b76-c4af68562b34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a8bcbc8381e4a769208732b1bfb31c2', 'request_id': 'req-c38cdeb8-98b3-4454-9b76-c4af68562b34'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:14.875466 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c38cdeb8-98b3-4454-9b76-c4af68562b34 admin admin] The token cache is being invalidate because project 5a8bcbc8381e4a769208732b1bfb31c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:14.875794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c38cdeb8-98b3-4454-9b76-c4af68562b34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:14.876576 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c38cdeb8-98b3-4454-9b76-c4af68562b34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:14.878291 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1255/2537] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:14 2022] DELETE /identity/v3/projects/5a8bcbc8381e4a769208732b1bfb31c2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:14.883753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec391c7-eac9-4b2f-a9e3-b9c06cf64fb8 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:14.884251 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec391c7-eac9-4b2f-a9e3-b9c06cf64fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:14.884352 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec391c7-eac9-4b2f-a9e3-b9c06cf64fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:14.905024 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7ec391c7-eac9-4b2f-a9e3-b9c06cf64fb8 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:14.906388 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7ec391c7-eac9-4b2f-a9e3-b9c06cf64fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:14.969963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1283/2538] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:14.978063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2f0e8-acc0-4eb3-8f5a-8a01c779a871 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:14.978346 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2f0e8-acc0-4eb3-8f5a-8a01c779a871 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:14.978618 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2f0e8-acc0-4eb3-8f5a-8a01c779a871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:14.997762 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e6f2f0e8-acc0-4eb3-8f5a-8a01c779a871 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:14.998980 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e6f2f0e8-acc0-4eb3-8f5a-8a01c779a871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:15.007590 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1256/2539] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:15.012220 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec391c7-eac9-4b2f-a9e3-b9c06cf64fb8 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.022327 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec391c7-eac9-4b2f-a9e3-b9c06cf64fb8 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.029245 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3b86f-bd0f-45cb-9692-eacf7c5adb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.030260 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3b86f-bd0f-45cb-9692-eacf7c5adb66 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.030596 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3b86f-bd0f-45cb-9692-eacf7c5adb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.030952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3b86f-bd0f-45cb-9692-eacf7c5adb66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.032128 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3b86f-bd0f-45cb-9692-eacf7c5adb66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:23:15.032587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3b86f-bd0f-45cb-9692-eacf7c5adb66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.035122 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3b86f-bd0f-45cb-9692-eacf7c5adb66 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.044688 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:15.051098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1284/2540] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:23:15.055407 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2f0e8-acc0-4eb3-8f5a-8a01c779a871 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.064914 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2f0e8-acc0-4eb3-8f5a-8a01c779a871 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.069901 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f95721-a314-4c6a-87ed-aa09b6b011ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.070551 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f95721-a314-4c6a-87ed-aa09b6b011ba admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.070833 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f95721-a314-4c6a-87ed-aa09b6b011ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.071109 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f95721-a314-4c6a-87ed-aa09b6b011ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.071832 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f95721-a314-4c6a-87ed-aa09b6b011ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.073867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f95721-a314-4c6a-87ed-aa09b6b011ba admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.100107 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1257/2541] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:15 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:15.103978 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3b86f-bd0f-45cb-9692-eacf7c5adb66 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.114725 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3b86f-bd0f-45cb-9692-eacf7c5adb66 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.181499 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de978df-d712-4bf4-923d-c97e443e03d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.182199 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de978df-d712-4bf4-923d-c97e443e03d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.182476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de978df-d712-4bf4-923d-c97e443e03d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.182742 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de978df-d712-4bf4-923d-c97e443e03d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.183423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de978df-d712-4bf4-923d-c97e443e03d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.185402 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de978df-d712-4bf4-923d-c97e443e03d3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.207016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1285/2542] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:15 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:15.211258 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f95721-a314-4c6a-87ed-aa09b6b011ba admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.221571 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f95721-a314-4c6a-87ed-aa09b6b011ba admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.226765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a98b92-f243-4b8b-866a-8d7e18ee78b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.227454 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a98b92-f243-4b8b-866a-8d7e18ee78b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.227765 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a98b92-f243-4b8b-866a-8d7e18ee78b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.228039 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a98b92-f243-4b8b-866a-8d7e18ee78b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.228755 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a98b92-f243-4b8b-866a-8d7e18ee78b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.230381 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a98b92-f243-4b8b-866a-8d7e18ee78b6 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.241597 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1258/2543] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:15.245621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de978df-d712-4bf4-923d-c97e443e03d3 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.256329 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de978df-d712-4bf4-923d-c97e443e03d3 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.261653 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26bf24-b199-4f62-8a06-3fb3adc495c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.262312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26bf24-b199-4f62-8a06-3fb3adc495c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.262605 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26bf24-b199-4f62-8a06-3fb3adc495c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.262869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26bf24-b199-4f62-8a06-3fb3adc495c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.263596 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b26bf24-b199-4f62-8a06-3fb3adc495c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.265225 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b26bf24-b199-4f62-8a06-3fb3adc495c2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.275112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1286/2544] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:15.279317 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a98b92-f243-4b8b-866a-8d7e18ee78b6 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.289696 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a98b92-f243-4b8b-866a-8d7e18ee78b6 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.295060 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570300e-fed0-4cfc-8229-955692ae36b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.295908 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570300e-fed0-4cfc-8229-955692ae36b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.296189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570300e-fed0-4cfc-8229-955692ae36b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.296428 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570300e-fed0-4cfc-8229-955692ae36b8 admin admin] PATH_INFO: `/v3/projects/0bd09f941d3b433f8c6c39e80fb0f3e9/users/4e71a30c493d4451afd7bbb46d40d00d/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.311720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570300e-fed0-4cfc-8229-955692ae36b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd09f941d3b433f8c6c39e80fb0f3e9, user_id=4e71a30c493d4451afd7bbb46d40d00d, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.314396 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570300e-fed0-4cfc-8229-955692ae36b8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.329256 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1259/2545] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:15 2022] PUT /identity/v3/projects/0bd09f941d3b433f8c6c39e80fb0f3e9/users/4e71a30c493d4451afd7bbb46d40d00d/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:15.341970 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6aab8b-3f51-4bcd-a6a0-52c8b26bb4ae None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.342289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6aab8b-3f51-4bcd-a6a0-52c8b26bb4ae None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.342710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6aab8b-3f51-4bcd-a6a0-52c8b26bb4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.375579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-fd6aab8b-3f51-4bcd-a6a0-52c8b26bb4ae None None] MFA Rules not processed for user `4e71a30c493d4451afd7bbb46d40d00d`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:15.376965 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fd6aab8b-3f51-4bcd-a6a0-52c8b26bb4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:15.444660 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1287/2546] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:15.449169 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570300e-fed0-4cfc-8229-955692ae36b8 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.484202 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:15.522176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570300e-fed0-4cfc-8229-955692ae36b8 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.527111 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7abbb-63b3-4671-b9bb-e165c17b135f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.527834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7abbb-63b3-4671-b9bb-e165c17b135f admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.528115 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7abbb-63b3-4671-b9bb-e165c17b135f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.528393 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7abbb-63b3-4671-b9bb-e165c17b135f admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.529090 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7abbb-63b3-4671-b9bb-e165c17b135f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.531042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7abbb-63b3-4671-b9bb-e165c17b135f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.539656 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6aab8b-3f51-4bcd-a6a0-52c8b26bb4ae None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.541234 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fd6aab8b-3f51-4bcd-a6a0-52c8b26bb4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:15.555434 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6aab8b-3f51-4bcd-a6a0-52c8b26bb4ae None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.559403 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1260/2547] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:15 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:15.560214 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d231b-fdbc-4068-b8b2-1f47b94d70f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.561149 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d231b-fdbc-4068-b8b2-1f47b94d70f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.561428 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d231b-fdbc-4068-b8b2-1f47b94d70f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.561700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d231b-fdbc-4068-b8b2-1f47b94d70f1 admin admin] PATH_INFO: `/v3/users/239aab5533be4cbfa995c0015cb4af5c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.562947 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d231b-fdbc-4068-b8b2-1f47b94d70f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=239aab5533be4cbfa995c0015cb4af5c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.563442 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7abbb-63b3-4671-b9bb-e165c17b135f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.564949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d231b-fdbc-4068-b8b2-1f47b94d70f1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.614313 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-355d231b-fdbc-4068-b8b2-1f47b94d70f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '239aab5533be4cbfa995c0015cb4af5c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:15.615026 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-355d231b-fdbc-4068-b8b2-1f47b94d70f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '239aab5533be4cbfa995c0015cb4af5c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:15.615795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-355d231b-fdbc-4068-b8b2-1f47b94d70f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '239aab5533be4cbfa995c0015cb4af5c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:15.626771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:15.629481 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7abbb-63b3-4671-b9bb-e165c17b135f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.629925 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-355d231b-fdbc-4068-b8b2-1f47b94d70f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '239aab5533be4cbfa995c0015cb4af5c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:15.635125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8d001-4cab-45e2-a0cc-62c595280af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.635844 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8d001-4cab-45e2-a0cc-62c595280af2 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.636131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8d001-4cab-45e2-a0cc-62c595280af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.636405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8d001-4cab-45e2-a0cc-62c595280af2 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.637422 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8d001-4cab-45e2-a0cc-62c595280af2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.639275 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8d001-4cab-45e2-a0cc-62c595280af2 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.639509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-355d231b-fdbc-4068-b8b2-1f47b94d70f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '239aab5533be4cbfa995c0015cb4af5c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:15.653478 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1288/2548] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:15 2022] DELETE /identity/v3/users/239aab5533be4cbfa995c0015cb4af5c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:15.658097 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74549f-f0a4-482d-8a49-426fdc7e78e8 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.658379 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74549f-f0a4-482d-8a49-426fdc7e78e8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.658805 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74549f-f0a4-482d-8a49-426fdc7e78e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.665008 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1261/2549] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:15 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:15.665952 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8d001-4cab-45e2-a0cc-62c595280af2 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.667671 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c1d8d001-4cab-45e2-a0cc-62c595280af2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:15.676196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-bd74549f-f0a4-482d-8a49-426fdc7e78e8 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:15.677476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bd74549f-f0a4-482d-8a49-426fdc7e78e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:15.680695 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8d001-4cab-45e2-a0cc-62c595280af2 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.737215 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114cea8d-509d-4d62-b284-a9efcaea8a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.737934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cea8d-509d-4d62-b284-a9efcaea8a9f admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.738170 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cea8d-509d-4d62-b284-a9efcaea8a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.738447 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cea8d-509d-4d62-b284-a9efcaea8a9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.739181 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114cea8d-509d-4d62-b284-a9efcaea8a9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.739451 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1289/2550] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:15.741010 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114cea8d-509d-4d62-b284-a9efcaea8a9f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.749116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74549f-f0a4-482d-8a49-426fdc7e78e8 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.750575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bd74549f-f0a4-482d-8a49-426fdc7e78e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:15.752382 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1262/2551] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:23:15.756596 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114cea8d-509d-4d62-b284-a9efcaea8a9f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.766881 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114cea8d-509d-4d62-b284-a9efcaea8a9f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.772797 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef025896-5705-4763-87e7-dbc9f3e388cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.773751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef025896-5705-4763-87e7-dbc9f3e388cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.774126 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef025896-5705-4763-87e7-dbc9f3e388cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.774476 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef025896-5705-4763-87e7-dbc9f3e388cc admin admin] PATH_INFO: `/v3/projects/d4cee761fbae427c8960719746425e2c/users/33e5d7bb6bd145c19824aedfc6ab4d18/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.791303 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef025896-5705-4763-87e7-dbc9f3e388cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4cee761fbae427c8960719746425e2c, user_id=33e5d7bb6bd145c19824aedfc6ab4d18, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.794272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef025896-5705-4763-87e7-dbc9f3e388cc admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.807680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74549f-f0a4-482d-8a49-426fdc7e78e8 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.808088 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1263/2552] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:15 2022] PUT /identity/v3/projects/d4cee761fbae427c8960719746425e2c/users/33e5d7bb6bd145c19824aedfc6ab4d18/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:15.812091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef025896-5705-4763-87e7-dbc9f3e388cc admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.829841 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:15.858832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc8a37-e755-4dda-a54a-b311e4bc42fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.859479 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc8a37-e755-4dda-a54a-b311e4bc42fe service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.859791 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc8a37-e755-4dda-a54a-b311e4bc42fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.860064 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc8a37-e755-4dda-a54a-b311e4bc42fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.879209 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef025896-5705-4763-87e7-dbc9f3e388cc admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.884140 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba66b790-cf93-4d5c-a3c1-108a8a149547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.884804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba66b790-cf93-4d5c-a3c1-108a8a149547 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.885082 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba66b790-cf93-4d5c-a3c1-108a8a149547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.885353 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba66b790-cf93-4d5c-a3c1-108a8a149547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.886223 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba66b790-cf93-4d5c-a3c1-108a8a149547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.887853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba66b790-cf93-4d5c-a3c1-108a8a149547 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.898368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1264/2553] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:23:15.909040 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba66b790-cf93-4d5c-a3c1-108a8a149547 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.917414 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:15.920936 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba66b790-cf93-4d5c-a3c1-108a8a149547 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.926441 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f2515-eefb-404d-af2e-ccdc78fd35be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.927525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f2515-eefb-404d-af2e-ccdc78fd35be admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.927832 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f2515-eefb-404d-af2e-ccdc78fd35be admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.928240 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f2515-eefb-404d-af2e-ccdc78fd35be admin admin] PATH_INFO: `/v3/projects/d4cee761fbae427c8960719746425e2c/users/33e5d7bb6bd145c19824aedfc6ab4d18/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.930214 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f2515-eefb-404d-af2e-ccdc78fd35be admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4cee761fbae427c8960719746425e2c, user_id=33e5d7bb6bd145c19824aedfc6ab4d18, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.933040 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f2515-eefb-404d-af2e-ccdc78fd35be admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.935387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc8a37-e755-4dda-a54a-b311e4bc42fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.936933 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc8a37-e755-4dda-a54a-b311e4bc42fe service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.943781 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1265/2554] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:15 2022] PUT /identity/v3/projects/d4cee761fbae427c8960719746425e2c/users/33e5d7bb6bd145c19824aedfc6ab4d18/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:15.950790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1290/2555] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:23:15.954212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65b0cf-5248-4262-b205-a3decc676442 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.954568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65b0cf-5248-4262-b205-a3decc676442 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.954914 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65b0cf-5248-4262-b205-a3decc676442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.983464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-7c65b0cf-5248-4262-b205-a3decc676442 None None] MFA Rules not processed for user `33e5d7bb6bd145c19824aedfc6ab4d18`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:15.984761 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7c65b0cf-5248-4262-b205-a3decc676442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:16.060424 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1266/2556] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:16.071641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc8a37-e755-4dda-a54a-b311e4bc42fe service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:16.073122 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cfdc8a37-e755-4dda-a54a-b311e4bc42fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:16.136265 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc8a37-e755-4dda-a54a-b311e4bc42fe service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:16.191555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172463a-fbc8-463a-893c-ac1a0928f021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:16.192281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172463a-fbc8-463a-893c-ac1a0928f021 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:16.192564 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172463a-fbc8-463a-893c-ac1a0928f021 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:16.192853 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172463a-fbc8-463a-893c-ac1a0928f021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:16.208993 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172463a-fbc8-463a-893c-ac1a0928f021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:16.211220 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172463a-fbc8-463a-893c-ac1a0928f021 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:16.224428 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1291/2557] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:16.241308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65b0cf-5248-4262-b205-a3decc676442 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:16.244224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7c65b0cf-5248-4262-b205-a3decc676442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:16.298422 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172463a-fbc8-463a-893c-ac1a0928f021 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:16.334397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65b0cf-5248-4262-b205-a3decc676442 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:16.338944 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92c380-af9a-41fd-a22c-5fcef647f223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:16.339865 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92c380-af9a-41fd-a22c-5fcef647f223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:16.340141 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92c380-af9a-41fd-a22c-5fcef647f223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:16.340415 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92c380-af9a-41fd-a22c-5fcef647f223 admin admin] PATH_INFO: `/v3/projects/93711c84abb7447aa9951d7bdb78df57` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:16.341587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92c380-af9a-41fd-a22c-5fcef647f223 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93711c84abb7447aa9951d7bdb78df57)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:16.343481 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92c380-af9a-41fd-a22c-5fcef647f223 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:16.361939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172463a-fbc8-463a-893c-ac1a0928f021 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:16.366600 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37846dd0-e1af-4d6c-a510-aa5827f2fbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:16.367569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37846dd0-e1af-4d6c-a510-aa5827f2fbd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:16.367887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37846dd0-e1af-4d6c-a510-aa5827f2fbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:16.368164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37846dd0-e1af-4d6c-a510-aa5827f2fbd4 admin admin] PATH_INFO: `/v3/users/4e71a30c493d4451afd7bbb46d40d00d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:16.369505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37846dd0-e1af-4d6c-a510-aa5827f2fbd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e71a30c493d4451afd7bbb46d40d00d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:16.371509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37846dd0-e1af-4d6c-a510-aa5827f2fbd4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:16.374417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:16.392807 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1d92c380-af9a-41fd-a22c-5fcef647f223 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93711c84abb7447aa9951d7bdb78df57', 'request_id': 'req-1d92c380-af9a-41fd-a22c-5fcef647f223'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:16.401864 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1d92c380-af9a-41fd-a22c-5fcef647f223 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93711c84abb7447aa9951d7bdb78df57', 'request_id': 'req-1d92c380-af9a-41fd-a22c-5fcef647f223'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:16.408316 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1d92c380-af9a-41fd-a22c-5fcef647f223 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93711c84abb7447aa9951d7bdb78df57', 'request_id': 'req-1d92c380-af9a-41fd-a22c-5fcef647f223'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:16.419756 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:16.420303 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1d92c380-af9a-41fd-a22c-5fcef647f223 admin admin] The token cache is being invalidate because project 93711c84abb7447aa9951d7bdb78df57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:16.420602 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1d92c380-af9a-41fd-a22c-5fcef647f223 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:16.421453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1d92c380-af9a-41fd-a22c-5fcef647f223 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:16.421878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-37846dd0-e1af-4d6c-a510-aa5827f2fbd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e71a30c493d4451afd7bbb46d40d00d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:16.422723 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-37846dd0-e1af-4d6c-a510-aa5827f2fbd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e71a30c493d4451afd7bbb46d40d00d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:16.423022 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1267/2558] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:16 2022] DELETE /identity/v3/projects/93711c84abb7447aa9951d7bdb78df57 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:16.426918 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92c380-af9a-41fd-a22c-5fcef647f223 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:16.427091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-37846dd0-e1af-4d6c-a510-aa5827f2fbd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e71a30c493d4451afd7bbb46d40d00d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:16.428443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1d92c380-af9a-41fd-a22c-5fcef647f223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:16.441056 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-37846dd0-e1af-4d6c-a510-aa5827f2fbd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e71a30c493d4451afd7bbb46d40d00d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:16.449062 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-37846dd0-e1af-4d6c-a510-aa5827f2fbd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e71a30c493d4451afd7bbb46d40d00d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:16.463209 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1292/2559] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:16 2022] DELETE /identity/v3/users/4e71a30c493d4451afd7bbb46d40d00d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:16.467966 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae986f0-b1f1-4a2c-9410-9434af822fd8 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:16.468257 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae986f0-b1f1-4a2c-9410-9434af822fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:16.468536 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae986f0-b1f1-4a2c-9410-9434af822fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:16.486163 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5ae986f0-b1f1-4a2c-9410-9434af822fd8 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:16.487531 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5ae986f0-b1f1-4a2c-9410-9434af822fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:16.498932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92c380-af9a-41fd-a22c-5fcef647f223 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:16.551144 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1293/2560] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:16.559672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413b050-dd90-4ddc-abd7-398a64ac4a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:16.560387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae986f0-b1f1-4a2c-9410-9434af822fd8 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:16.560643 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413b050-dd90-4ddc-abd7-398a64ac4a93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:16.560922 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413b050-dd90-4ddc-abd7-398a64ac4a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:16.561199 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413b050-dd90-4ddc-abd7-398a64ac4a93 admin admin] PATH_INFO: `/v3/users/4053a42eb22e42c6905c31041ab7e5fa` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:16.561808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5ae986f0-b1f1-4a2c-9410-9434af822fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:16.562416 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b413b050-dd90-4ddc-abd7-398a64ac4a93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4053a42eb22e42c6905c31041ab7e5fa)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:16.564445 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b413b050-dd90-4ddc-abd7-398a64ac4a93 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:16.613690 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b413b050-dd90-4ddc-abd7-398a64ac4a93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4053a42eb22e42c6905c31041ab7e5fa'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:16.614351 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b413b050-dd90-4ddc-abd7-398a64ac4a93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4053a42eb22e42c6905c31041ab7e5fa'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:16.614974 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b413b050-dd90-4ddc-abd7-398a64ac4a93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4053a42eb22e42c6905c31041ab7e5fa'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:16.619309 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae986f0-b1f1-4a2c-9410-9434af822fd8 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:16.623099 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a08ae2-7fe5-4059-bc8e-24a3e1cf13bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:16.623782 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a08ae2-7fe5-4059-bc8e-24a3e1cf13bc service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:16.624065 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a08ae2-7fe5-4059-bc8e-24a3e1cf13bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:16.624338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a08ae2-7fe5-4059-bc8e-24a3e1cf13bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:16.626027 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:16.626148 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c9a08ae2-7fe5-4059-bc8e-24a3e1cf13bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:16.627457 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b413b050-dd90-4ddc-abd7-398a64ac4a93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4053a42eb22e42c6905c31041ab7e5fa'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:16.637073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b413b050-dd90-4ddc-abd7-398a64ac4a93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4053a42eb22e42c6905c31041ab7e5fa'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:16.638940 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a08ae2-7fe5-4059-bc8e-24a3e1cf13bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:16.640435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a08ae2-7fe5-4059-bc8e-24a3e1cf13bc service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:16.650658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1294/2561] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:23:16.652550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1268/2562] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:16 2022] DELETE /identity/v3/users/4053a42eb22e42c6905c31041ab7e5fa => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:17.005444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a08ae2-7fe5-4059-bc8e-24a3e1cf13bc service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:17.007515 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c9a08ae2-7fe5-4059-bc8e-24a3e1cf13bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:17.012561 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413b050-dd90-4ddc-abd7-398a64ac4a93 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:17.013994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b413b050-dd90-4ddc-abd7-398a64ac4a93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:17.020775 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a08ae2-7fe5-4059-bc8e-24a3e1cf13bc service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:17.085258 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0b0ca-00c8-478d-8519-07fb17c79782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:17.086298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0b0ca-00c8-478d-8519-07fb17c79782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:17.086581 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0b0ca-00c8-478d-8519-07fb17c79782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:17.086917 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0b0ca-00c8-478d-8519-07fb17c79782 admin admin] PATH_INFO: `/v3/projects/0bd09f941d3b433f8c6c39e80fb0f3e9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:17.088169 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0b0ca-00c8-478d-8519-07fb17c79782 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bd09f941d3b433f8c6c39e80fb0f3e9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:17.090800 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0b0ca-00c8-478d-8519-07fb17c79782 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:17.098867 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:17.105442 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413b050-dd90-4ddc-abd7-398a64ac4a93 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:17.110459 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e95539-7a92-4a76-adc0-5bd8f4870034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:17.111366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e95539-7a92-4a76-adc0-5bd8f4870034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:17.111677 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e95539-7a92-4a76-adc0-5bd8f4870034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:17.111964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e95539-7a92-4a76-adc0-5bd8f4870034 admin admin] PATH_INFO: `/v3/projects/5ec3bb78af3e4868afef2863a5568ba5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:17.113148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e95539-7a92-4a76-adc0-5bd8f4870034 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ec3bb78af3e4868afef2863a5568ba5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:17.115057 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e95539-7a92-4a76-adc0-5bd8f4870034 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:17.140628 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-00c0b0ca-00c8-478d-8519-07fb17c79782 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bd09f941d3b433f8c6c39e80fb0f3e9', 'request_id': 'req-00c0b0ca-00c8-478d-8519-07fb17c79782'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:17.148657 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-00c0b0ca-00c8-478d-8519-07fb17c79782 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bd09f941d3b433f8c6c39e80fb0f3e9', 'request_id': 'req-00c0b0ca-00c8-478d-8519-07fb17c79782'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:17.154289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-00c0b0ca-00c8-478d-8519-07fb17c79782 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bd09f941d3b433f8c6c39e80fb0f3e9', 'request_id': 'req-00c0b0ca-00c8-478d-8519-07fb17c79782'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:17.164097 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-94e95539-7a92-4a76-adc0-5bd8f4870034 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ec3bb78af3e4868afef2863a5568ba5', 'request_id': 'req-94e95539-7a92-4a76-adc0-5bd8f4870034'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:17.165945 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-00c0b0ca-00c8-478d-8519-07fb17c79782 admin admin] The token cache is being invalidate because project 0bd09f941d3b433f8c6c39e80fb0f3e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:17.166227 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-00c0b0ca-00c8-478d-8519-07fb17c79782 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:17.166885 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-00c0b0ca-00c8-478d-8519-07fb17c79782 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:17.168518 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1295/2563] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:17 2022] DELETE /identity/v3/projects/0bd09f941d3b433f8c6c39e80fb0f3e9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:17.172733 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-94e95539-7a92-4a76-adc0-5bd8f4870034 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ec3bb78af3e4868afef2863a5568ba5', 'request_id': 'req-94e95539-7a92-4a76-adc0-5bd8f4870034'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:17.172912 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0b0ca-00c8-478d-8519-07fb17c79782 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:17.174268 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-00c0b0ca-00c8-478d-8519-07fb17c79782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:17.180201 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-94e95539-7a92-4a76-adc0-5bd8f4870034 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ec3bb78af3e4868afef2863a5568ba5', 'request_id': 'req-94e95539-7a92-4a76-adc0-5bd8f4870034'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:17.191982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-94e95539-7a92-4a76-adc0-5bd8f4870034 admin admin] The token cache is being invalidate because project 5ec3bb78af3e4868afef2863a5568ba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:17.192273 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-94e95539-7a92-4a76-adc0-5bd8f4870034 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:17.192998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-94e95539-7a92-4a76-adc0-5bd8f4870034 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:17.194538 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1269/2564] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:17 2022] DELETE /identity/v3/projects/5ec3bb78af3e4868afef2863a5568ba5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:17.199080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec97203-022b-4e48-bf4f-697e59b2e020 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:17.199362 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec97203-022b-4e48-bf4f-697e59b2e020 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:17.199681 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec97203-022b-4e48-bf4f-697e59b2e020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:17.217737 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-0ec97203-022b-4e48-bf4f-697e59b2e020 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:17.218951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0ec97203-022b-4e48-bf4f-697e59b2e020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:17.241301 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0b0ca-00c8-478d-8519-07fb17c79782 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:17.285442 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1270/2565] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:17.294398 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb35e43-ed6b-4681-9c98-3f92b0f3a9f3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:17.294715 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb35e43-ed6b-4681-9c98-3f92b0f3a9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:17.294996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb35e43-ed6b-4681-9c98-3f92b0f3a9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:17.298919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e7073-f3cf-4f2b-af01-3ca8df67c460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:17.299836 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e7073-f3cf-4f2b-af01-3ca8df67c460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:17.300114 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e7073-f3cf-4f2b-af01-3ca8df67c460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:17.300394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e7073-f3cf-4f2b-af01-3ca8df67c460 admin admin] PATH_INFO: `/v3/users/33e5d7bb6bd145c19824aedfc6ab4d18` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:17.301568 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e7073-f3cf-4f2b-af01-3ca8df67c460 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33e5d7bb6bd145c19824aedfc6ab4d18)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:17.303542 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e7073-f3cf-4f2b-af01-3ca8df67c460 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:17.313021 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6fb35e43-ed6b-4681-9c98-3f92b0f3a9f3 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:17.314237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6fb35e43-ed6b-4681-9c98-3f92b0f3a9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:17.322658 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1271/2566] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:17.326967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb35e43-ed6b-4681-9c98-3f92b0f3a9f3 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:17.333294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:17.338071 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb35e43-ed6b-4681-9c98-3f92b0f3a9f3 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:17.345796 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9cbe8-1f00-4abf-a10d-dd7f3d0c2161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:17.346682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9cbe8-1f00-4abf-a10d-dd7f3d0c2161 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:17.347215 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9cbe8-1f00-4abf-a10d-dd7f3d0c2161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:17.347488 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9cbe8-1f00-4abf-a10d-dd7f3d0c2161 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:17.348242 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9cbe8-1f00-4abf-a10d-dd7f3d0c2161 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:23:17.348551 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9cbe8-1f00-4abf-a10d-dd7f3d0c2161 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:17.349665 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:17.350150 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9cbe8-1f00-4abf-a10d-dd7f3d0c2161 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:17.351700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fa6e7073-f3cf-4f2b-af01-3ca8df67c460 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33e5d7bb6bd145c19824aedfc6ab4d18'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:17.352701 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fa6e7073-f3cf-4f2b-af01-3ca8df67c460 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33e5d7bb6bd145c19824aedfc6ab4d18'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:17.353651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fa6e7073-f3cf-4f2b-af01-3ca8df67c460 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33e5d7bb6bd145c19824aedfc6ab4d18'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:17.362085 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1272/2567] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:23:17.366737 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9cbe8-1f00-4abf-a10d-dd7f3d0c2161 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:17.367676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fa6e7073-f3cf-4f2b-af01-3ca8df67c460 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33e5d7bb6bd145c19824aedfc6ab4d18'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:17.368577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c1f9cbe8-1f00-4abf-a10d-dd7f3d0c2161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:17.374942 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fa6e7073-f3cf-4f2b-af01-3ca8df67c460 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33e5d7bb6bd145c19824aedfc6ab4d18'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:17.381938 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9cbe8-1f00-4abf-a10d-dd7f3d0c2161 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:17.387014 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72435cdf-204d-4266-a4b5-c4d6c8bb74e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:17.387718 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72435cdf-204d-4266-a4b5-c4d6c8bb74e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:17.388009 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72435cdf-204d-4266-a4b5-c4d6c8bb74e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:17.388291 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72435cdf-204d-4266-a4b5-c4d6c8bb74e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:17.389112 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72435cdf-204d-4266-a4b5-c4d6c8bb74e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:17.389254 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1296/2568] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:17 2022] DELETE /identity/v3/users/33e5d7bb6bd145c19824aedfc6ab4d18 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:17.391171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72435cdf-204d-4266-a4b5-c4d6c8bb74e9 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:17.421647 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1273/2569] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:17 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:17.423145 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e7073-f3cf-4f2b-af01-3ca8df67c460 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:17.488903 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e7073-f3cf-4f2b-af01-3ca8df67c460 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:17.565513 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bc736-cce1-4f24-b608-0cc443288892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:17.566204 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bc736-cce1-4f24-b608-0cc443288892 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:17.566488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bc736-cce1-4f24-b608-0cc443288892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:17.566742 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bc736-cce1-4f24-b608-0cc443288892 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:17.567470 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49bc736-cce1-4f24-b608-0cc443288892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:17.569519 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49bc736-cce1-4f24-b608-0cc443288892 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:17.593346 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1297/2570] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:17 2022] POST /identity/v3/users => generated 292 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:17.594967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72435cdf-204d-4266-a4b5-c4d6c8bb74e9 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:17.606104 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72435cdf-204d-4266-a4b5-c4d6c8bb74e9 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:17.611156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97facaf2-6a0f-4806-884f-58309982ef48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:17.612046 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97facaf2-6a0f-4806-884f-58309982ef48 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:17.612402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97facaf2-6a0f-4806-884f-58309982ef48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:17.612873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97facaf2-6a0f-4806-884f-58309982ef48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:17.613492 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97facaf2-6a0f-4806-884f-58309982ef48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:17.615097 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97facaf2-6a0f-4806-884f-58309982ef48 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:17.625718 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1274/2571] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:17.629870 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bc736-cce1-4f24-b608-0cc443288892 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:17.641308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bc736-cce1-4f24-b608-0cc443288892 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:17.646570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb507b-696e-4546-81b6-4bcaff8e12ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:17.647337 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb507b-696e-4546-81b6-4bcaff8e12ff admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:17.647723 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb507b-696e-4546-81b6-4bcaff8e12ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:17.648065 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb507b-696e-4546-81b6-4bcaff8e12ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:17.648864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fb507b-696e-4546-81b6-4bcaff8e12ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:17.650597 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fb507b-696e-4546-81b6-4bcaff8e12ff admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:17.663059 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1298/2572] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:17.667824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97facaf2-6a0f-4806-884f-58309982ef48 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:17.675760 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb507b-696e-4546-81b6-4bcaff8e12ff admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:17.678006 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-50fb507b-696e-4546-81b6-4bcaff8e12ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:17.679855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97facaf2-6a0f-4806-884f-58309982ef48 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:17.684879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82204785-482e-4fdb-8f16-7238ab5f206b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:17.685750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82204785-482e-4fdb-8f16-7238ab5f206b admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:17.686044 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82204785-482e-4fdb-8f16-7238ab5f206b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:17.686278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82204785-482e-4fdb-8f16-7238ab5f206b admin admin] PATH_INFO: `/v3/projects/f74199d04bc04885855dbbf7b8457d1e/users/c31ccf9915c94bd59dcecd4a310e15a9/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:17.697045 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb507b-696e-4546-81b6-4bcaff8e12ff admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:17.702305 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb47082-3b55-47b4-9fcf-60a38b469073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:17.703352 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb47082-3b55-47b4-9fcf-60a38b469073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:17.703629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb47082-3b55-47b4-9fcf-60a38b469073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:17.703898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb47082-3b55-47b4-9fcf-60a38b469073 admin admin] PATH_INFO: `/v3/projects/d4cee761fbae427c8960719746425e2c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:17.704016 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82204785-482e-4fdb-8f16-7238ab5f206b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f74199d04bc04885855dbbf7b8457d1e, user_id=c31ccf9915c94bd59dcecd4a310e15a9, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:17.705111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb47082-3b55-47b4-9fcf-60a38b469073 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4cee761fbae427c8960719746425e2c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:17.706563 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82204785-482e-4fdb-8f16-7238ab5f206b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:17.707031 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb47082-3b55-47b4-9fcf-60a38b469073 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:17.721618 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1275/2573] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:17 2022] PUT /identity/v3/projects/f74199d04bc04885855dbbf7b8457d1e/users/c31ccf9915c94bd59dcecd4a310e15a9/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:17.732155 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97799e4-4a4f-4cdc-99cb-121a38dc21d4 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:17.732442 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97799e4-4a4f-4cdc-99cb-121a38dc21d4 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:17.732718 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97799e4-4a4f-4cdc-99cb-121a38dc21d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:17.755018 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:17.758677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1fb47082-3b55-47b4-9fcf-60a38b469073 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4cee761fbae427c8960719746425e2c', 'request_id': 'req-1fb47082-3b55-47b4-9fcf-60a38b469073'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:17.761358 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a97799e4-4a4f-4cdc-99cb-121a38dc21d4 None None] MFA Rules not processed for user `c31ccf9915c94bd59dcecd4a310e15a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:17.762701 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a97799e4-4a4f-4cdc-99cb-121a38dc21d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:17.768341 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1fb47082-3b55-47b4-9fcf-60a38b469073 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4cee761fbae427c8960719746425e2c', 'request_id': 'req-1fb47082-3b55-47b4-9fcf-60a38b469073'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:17.774053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1fb47082-3b55-47b4-9fcf-60a38b469073 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4cee761fbae427c8960719746425e2c', 'request_id': 'req-1fb47082-3b55-47b4-9fcf-60a38b469073'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:17.789009 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1fb47082-3b55-47b4-9fcf-60a38b469073 admin admin] The token cache is being invalidate because project d4cee761fbae427c8960719746425e2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:17.789509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1fb47082-3b55-47b4-9fcf-60a38b469073 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:17.790562 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1fb47082-3b55-47b4-9fcf-60a38b469073 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:17.792585 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1299/2574] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:17 2022] DELETE /identity/v3/projects/d4cee761fbae427c8960719746425e2c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:17.798702 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5853e60-3d80-4224-90d8-c0bce19b8a07 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:17.799063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5853e60-3d80-4224-90d8-c0bce19b8a07 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:17.799335 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5853e60-3d80-4224-90d8-c0bce19b8a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:17.817155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c5853e60-3d80-4224-90d8-c0bce19b8a07 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:17.818501 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c5853e60-3d80-4224-90d8-c0bce19b8a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:17.831958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1276/2575] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:17.837430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97799e4-4a4f-4cdc-99cb-121a38dc21d4 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:17.839486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a97799e4-4a4f-4cdc-99cb-121a38dc21d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:17.881105 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1300/2576] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:17.888818 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205c63a-417a-450c-81f0-5d524439aa32 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:17.889105 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205c63a-417a-450c-81f0-5d524439aa32 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:17.889466 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205c63a-417a-450c-81f0-5d524439aa32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:17.907342 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6205c63a-417a-450c-81f0-5d524439aa32 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:17.908507 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6205c63a-417a-450c-81f0-5d524439aa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:17.909003 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97799e4-4a4f-4cdc-99cb-121a38dc21d4 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:17.913874 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce450e5-fb4c-4447-bdcf-0f470e874db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:17.914536 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce450e5-fb4c-4447-bdcf-0f470e874db7 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:17.914820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce450e5-fb4c-4447-bdcf-0f470e874db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:17.915096 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce450e5-fb4c-4447-bdcf-0f470e874db7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:17.915809 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce450e5-fb4c-4447-bdcf-0f470e874db7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:17.916207 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1301/2577] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:17.917696 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce450e5-fb4c-4447-bdcf-0f470e874db7 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:17.920279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205c63a-417a-450c-81f0-5d524439aa32 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:17.929147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205c63a-417a-450c-81f0-5d524439aa32 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:17.933990 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e30eb-b850-4626-8489-9248fc5d8cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:17.934642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e30eb-b850-4626-8489-9248fc5d8cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:17.935019 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e30eb-b850-4626-8489-9248fc5d8cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:17.935170 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e30eb-b850-4626-8489-9248fc5d8cc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:17.935936 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e30eb-b850-4626-8489-9248fc5d8cc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:23:17.936243 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e30eb-b850-4626-8489-9248fc5d8cc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:17.937885 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e30eb-b850-4626-8489-9248fc5d8cc5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:17.945139 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1277/2578] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:17 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:17.949114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce450e5-fb4c-4447-bdcf-0f470e874db7 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:17.952570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1302/2579] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:23:17.953444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e30eb-b850-4626-8489-9248fc5d8cc5 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:17.959026 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:17.963178 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e30eb-b850-4626-8489-9248fc5d8cc5 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:18.014327 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce450e5-fb4c-4447-bdcf-0f470e874db7 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:18.019063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafeccf-f110-45bf-b026-6d214a09c36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:18.019810 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafeccf-f110-45bf-b026-6d214a09c36d admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.020083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafeccf-f110-45bf-b026-6d214a09c36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.020372 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafeccf-f110-45bf-b026-6d214a09c36d admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.021057 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafeccf-f110-45bf-b026-6d214a09c36d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:18.022990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafeccf-f110-45bf-b026-6d214a09c36d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:18.030976 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a156df2-7b52-4028-8325-0a1dd64c7564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:18.031658 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a156df2-7b52-4028-8325-0a1dd64c7564 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.031943 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a156df2-7b52-4028-8325-0a1dd64c7564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.032214 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a156df2-7b52-4028-8325-0a1dd64c7564 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.032894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a156df2-7b52-4028-8325-0a1dd64c7564 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:18.034832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a156df2-7b52-4028-8325-0a1dd64c7564 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:18.044203 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1278/2580] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:17 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:18.048067 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafeccf-f110-45bf-b026-6d214a09c36d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:18.057810 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafeccf-f110-45bf-b026-6d214a09c36d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:18.062441 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1303/2581] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:17 2022] POST /identity/v3/projects => generated 401 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:18.062590 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616b087-d853-4dac-92e2-82ed6dcaa7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:18.063002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616b087-d853-4dac-92e2-82ed6dcaa7eb admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.063287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616b087-d853-4dac-92e2-82ed6dcaa7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.063564 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616b087-d853-4dac-92e2-82ed6dcaa7eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.064507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7616b087-d853-4dac-92e2-82ed6dcaa7eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:18.066157 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7616b087-d853-4dac-92e2-82ed6dcaa7eb admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:18.066689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a156df2-7b52-4028-8325-0a1dd64c7564 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:18.079395 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1279/2582] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:18.080648 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616b087-d853-4dac-92e2-82ed6dcaa7eb admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:18.090166 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616b087-d853-4dac-92e2-82ed6dcaa7eb admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:18.106833 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:18.131067 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a156df2-7b52-4028-8325-0a1dd64c7564 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:18.136496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cd765-2f93-4f87-9764-284ea4618c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:18.137176 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cd765-2f93-4f87-9764-284ea4618c21 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.137618 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cd765-2f93-4f87-9764-284ea4618c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.137757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cd765-2f93-4f87-9764-284ea4618c21 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.138444 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0cd765-2f93-4f87-9764-284ea4618c21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:18.140623 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0cd765-2f93-4f87-9764-284ea4618c21 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:18.150864 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2068b-cce8-4959-bb0b-808d4140dae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:18.151823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2068b-cce8-4959-bb0b-808d4140dae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.152146 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2068b-cce8-4959-bb0b-808d4140dae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.152390 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2068b-cce8-4959-bb0b-808d4140dae1 admin admin] PATH_INFO: `/v3/projects/bcb1c17035d04e1887799ae3078f3345/users/40c66fa811414f5cbe1554958abadd4c/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.164198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1304/2583] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:18 2022] POST /identity/v3/users => generated 301 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:18.165191 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cd765-2f93-4f87-9764-284ea4618c21 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:18.167556 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2068b-cce8-4959-bb0b-808d4140dae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcb1c17035d04e1887799ae3078f3345, user_id=40c66fa811414f5cbe1554958abadd4c, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:18.170253 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2068b-cce8-4959-bb0b-808d4140dae1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:18.175696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cd765-2f93-4f87-9764-284ea4618c21 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:18.184621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636a15f-63c3-4bb5-a134-4da9a319ad6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:18.185300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636a15f-63c3-4bb5-a134-4da9a319ad6a admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.185579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636a15f-63c3-4bb5-a134-4da9a319ad6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.185849 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636a15f-63c3-4bb5-a134-4da9a319ad6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.186577 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636a15f-63c3-4bb5-a134-4da9a319ad6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:18.186716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1280/2584] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:18 2022] PUT /identity/v3/projects/bcb1c17035d04e1887799ae3078f3345/users/40c66fa811414f5cbe1554958abadd4c/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:18.188491 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636a15f-63c3-4bb5-a134-4da9a319ad6a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:18.190596 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2068b-cce8-4959-bb0b-808d4140dae1 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:18.200368 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1305/2585] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:23:18.204323 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636a15f-63c3-4bb5-a134-4da9a319ad6a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:18.214673 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636a15f-63c3-4bb5-a134-4da9a319ad6a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:18.254285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2068b-cce8-4959-bb0b-808d4140dae1 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:18.259117 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44d324-8b15-49fe-982a-e9a9d3a50237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:18.259820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44d324-8b15-49fe-982a-e9a9d3a50237 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.260106 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44d324-8b15-49fe-982a-e9a9d3a50237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.260439 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44d324-8b15-49fe-982a-e9a9d3a50237 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.261163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44d324-8b15-49fe-982a-e9a9d3a50237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:18.262778 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44d324-8b15-49fe-982a-e9a9d3a50237 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:18.271095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230f3ff-c508-4ecf-ae31-720176dcb8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:18.271822 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230f3ff-c508-4ecf-ae31-720176dcb8db admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.272106 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230f3ff-c508-4ecf-ae31-720176dcb8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.272380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230f3ff-c508-4ecf-ae31-720176dcb8db admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.273106 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230f3ff-c508-4ecf-ae31-720176dcb8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:18.273200 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1281/2586] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:23:18.274704 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230f3ff-c508-4ecf-ae31-720176dcb8db admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:18.277240 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44d324-8b15-49fe-982a-e9a9d3a50237 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:18.289213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1306/2587] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:23:18.289470 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44d324-8b15-49fe-982a-e9a9d3a50237 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:18.290552 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230f3ff-c508-4ecf-ae31-720176dcb8db admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:18.291746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b29138-97db-4280-b9a1-5cf583624b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:18.292634 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b29138-97db-4280-b9a1-5cf583624b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.292912 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b29138-97db-4280-b9a1-5cf583624b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.293185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b29138-97db-4280-b9a1-5cf583624b50 admin admin] PATH_INFO: `/v3/projects/bcb1c17035d04e1887799ae3078f3345/users/40c66fa811414f5cbe1554958abadd4c/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.295330 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b29138-97db-4280-b9a1-5cf583624b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcb1c17035d04e1887799ae3078f3345, user_id=40c66fa811414f5cbe1554958abadd4c, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:18.298050 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b29138-97db-4280-b9a1-5cf583624b50 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:18.300409 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:18.301343 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230f3ff-c508-4ecf-ae31-720176dcb8db admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:18.306954 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe7495-7e63-40dc-bb8a-eed00e1a74c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:18.307893 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe7495-7e63-40dc-bb8a-eed00e1a74c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.308181 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe7495-7e63-40dc-bb8a-eed00e1a74c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.308567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe7495-7e63-40dc-bb8a-eed00e1a74c8 admin admin] PATH_INFO: `/v3/projects/13fb1c6a91e64b42aa6677c5d0754b05/users/c899129975324d4798277f09b755792e/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.310811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1282/2588] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:18 2022] PUT /identity/v3/projects/bcb1c17035d04e1887799ae3078f3345/users/40c66fa811414f5cbe1554958abadd4c/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:18.321033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea584ef-5aa9-466f-8569-e16317414b68 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.321323 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea584ef-5aa9-466f-8569-e16317414b68 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.321633 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea584ef-5aa9-466f-8569-e16317414b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.324740 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe7495-7e63-40dc-bb8a-eed00e1a74c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13fb1c6a91e64b42aa6677c5d0754b05, user_id=c899129975324d4798277f09b755792e, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:18.327451 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe7495-7e63-40dc-bb8a-eed00e1a74c8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:18.341623 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1307/2589] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:18 2022] PUT /identity/v3/projects/13fb1c6a91e64b42aa6677c5d0754b05/users/c899129975324d4798277f09b755792e/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:18.349734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-eea584ef-5aa9-466f-8569-e16317414b68 None None] MFA Rules not processed for user `40c66fa811414f5cbe1554958abadd4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:18.350951 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-eea584ef-5aa9-466f-8569-e16317414b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:18.352824 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d2988-639f-4625-a5f7-cf70ba710dcc None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.353110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d2988-639f-4625-a5f7-cf70ba710dcc None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.353380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d2988-639f-4625-a5f7-cf70ba710dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.379797 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ea0d2988-639f-4625-a5f7-cf70ba710dcc None None] MFA Rules not processed for user `c899129975324d4798277f09b755792e`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:18.381158 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ea0d2988-639f-4625-a5f7-cf70ba710dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:18.419426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1283/2590] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:18.430428 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea584ef-5aa9-466f-8569-e16317414b68 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:18.432401 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-eea584ef-5aa9-466f-8569-e16317414b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:18.449537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1308/2591] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:18.450512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d2988-639f-4625-a5f7-cf70ba710dcc None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:18.468849 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:18.492130 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea584ef-5aa9-466f-8569-e16317414b68 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:18.496584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c534f-e094-4865-88a7-e14895e626f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:18.497527 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c534f-e094-4865-88a7-e14895e626f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.497807 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c534f-e094-4865-88a7-e14895e626f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.498083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c534f-e094-4865-88a7-e14895e626f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.508065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c534f-e094-4865-88a7-e14895e626f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:18.509496 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c534f-e094-4865-88a7-e14895e626f0 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:18.513867 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d2988-639f-4625-a5f7-cf70ba710dcc None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:18.518635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89cd3e-c002-4666-b7ff-c33ddc8036ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:18.518900 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1284/2592] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:23:18.519439 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89cd3e-c002-4666-b7ff-c33ddc8036ef admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.519754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89cd3e-c002-4666-b7ff-c33ddc8036ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.520036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89cd3e-c002-4666-b7ff-c33ddc8036ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.520696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89cd3e-c002-4666-b7ff-c33ddc8036ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:18.522620 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89cd3e-c002-4666-b7ff-c33ddc8036ef admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:18.551113 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1309/2593] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:18 2022] POST /identity/v3/projects => generated 401 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:18.555077 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c534f-e094-4865-88a7-e14895e626f0 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:18.565903 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c534f-e094-4865-88a7-e14895e626f0 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:18.618505 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89cd3e-c002-4666-b7ff-c33ddc8036ef admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:18.624388 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed374138-5712-4609-bd04-d9bf0d070f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:18.624669 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed374138-5712-4609-bd04-d9bf0d070f74 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.624959 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed374138-5712-4609-bd04-d9bf0d070f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.625237 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed374138-5712-4609-bd04-d9bf0d070f74 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.625948 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed374138-5712-4609-bd04-d9bf0d070f74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:18.627927 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed374138-5712-4609-bd04-d9bf0d070f74 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:18.650924 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1285/2594] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:18 2022] POST /identity/v3/users => generated 307 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:18.652040 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed374138-5712-4609-bd04-d9bf0d070f74 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:18.661622 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed374138-5712-4609-bd04-d9bf0d070f74 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:18.666426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05778e-94a2-47d6-b86b-a9beecb8caf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:18.667137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05778e-94a2-47d6-b86b-a9beecb8caf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.667437 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05778e-94a2-47d6-b86b-a9beecb8caf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.667733 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05778e-94a2-47d6-b86b-a9beecb8caf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.668445 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05778e-94a2-47d6-b86b-a9beecb8caf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:18.670086 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05778e-94a2-47d6-b86b-a9beecb8caf1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:18.676017 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:18.678585 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89cd3e-c002-4666-b7ff-c33ddc8036ef admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:18.682044 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1286/2595] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:23:18.682366 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55550d4-64c5-42c2-b2e5-7ba5545c71a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:18.682900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55550d4-64c5-42c2-b2e5-7ba5545c71a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.683285 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55550d4-64c5-42c2-b2e5-7ba5545c71a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.683528 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55550d4-64c5-42c2-b2e5-7ba5545c71a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.685609 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05778e-94a2-47d6-b86b-a9beecb8caf1 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:18.695695 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05778e-94a2-47d6-b86b-a9beecb8caf1 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:18.700197 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b3d08-a595-46ba-8085-5c59dbdc1f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:18.701189 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b3d08-a595-46ba-8085-5c59dbdc1f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.701471 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b3d08-a595-46ba-8085-5c59dbdc1f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.701763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b3d08-a595-46ba-8085-5c59dbdc1f76 admin admin] PATH_INFO: `/v3/projects/e0af58b5da964b2aaa2bacffda5940b6/users/027ea69f071743c5ac36c19219ce44a2/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.716987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b3d08-a595-46ba-8085-5c59dbdc1f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0af58b5da964b2aaa2bacffda5940b6, user_id=027ea69f071743c5ac36c19219ce44a2, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:18.719663 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b3d08-a595-46ba-8085-5c59dbdc1f76 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:18.733844 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1287/2596] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:18 2022] PUT /identity/v3/projects/e0af58b5da964b2aaa2bacffda5940b6/users/027ea69f071743c5ac36c19219ce44a2/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:18.737869 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b3d08-a595-46ba-8085-5c59dbdc1f76 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:18.742432 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55550d4-64c5-42c2-b2e5-7ba5545c71a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:18.743937 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55550d4-64c5-42c2-b2e5-7ba5545c71a0 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:18.753046 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1310/2597] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:23:18.802225 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b3d08-a595-46ba-8085-5c59dbdc1f76 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:18.806934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d719bd3-3cc4-4a55-be87-bdccce9d5be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:18.807734 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d719bd3-3cc4-4a55-be87-bdccce9d5be8 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.808025 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d719bd3-3cc4-4a55-be87-bdccce9d5be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.808308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d719bd3-3cc4-4a55-be87-bdccce9d5be8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.809037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d719bd3-3cc4-4a55-be87-bdccce9d5be8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:18.810663 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d719bd3-3cc4-4a55-be87-bdccce9d5be8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:18.820953 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1288/2598] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:23:18.825675 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55550d4-64c5-42c2-b2e5-7ba5545c71a0 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:18.837994 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55550d4-64c5-42c2-b2e5-7ba5545c71a0 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:18.842756 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b56a1-f3e1-4a06-83e1-995a0ee4051b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:18.843691 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b56a1-f3e1-4a06-83e1-995a0ee4051b admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.843977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b56a1-f3e1-4a06-83e1-995a0ee4051b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.844255 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b56a1-f3e1-4a06-83e1-995a0ee4051b admin admin] PATH_INFO: `/v3/projects/e0af58b5da964b2aaa2bacffda5940b6/users/027ea69f071743c5ac36c19219ce44a2/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.846300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b56a1-f3e1-4a06-83e1-995a0ee4051b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0af58b5da964b2aaa2bacffda5940b6, user_id=027ea69f071743c5ac36c19219ce44a2, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:18.849016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b56a1-f3e1-4a06-83e1-995a0ee4051b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:18.860540 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1311/2599] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:18 2022] PUT /identity/v3/projects/e0af58b5da964b2aaa2bacffda5940b6/users/027ea69f071743c5ac36c19219ce44a2/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:18.871796 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726daffe-4969-4103-8ec2-d91e61a6532d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:18.872121 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726daffe-4969-4103-8ec2-d91e61a6532d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:18.872404 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726daffe-4969-4103-8ec2-d91e61a6532d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:18.901842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-726daffe-4969-4103-8ec2-d91e61a6532d None None] MFA Rules not processed for user `027ea69f071743c5ac36c19219ce44a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:18.903200 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-726daffe-4969-4103-8ec2-d91e61a6532d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:18.985193 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1289/2600] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:18 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:19.002565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b56a1-f3e1-4a06-83e1-995a0ee4051b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:19.004752 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3e6b56a1-f3e1-4a06-83e1-995a0ee4051b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:19.059636 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:19.072933 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b56a1-f3e1-4a06-83e1-995a0ee4051b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:19.077200 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6adc9-ab84-4cf8-b457-13849a254209 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:19.077883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6adc9-ab84-4cf8-b457-13849a254209 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:19.078150 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6adc9-ab84-4cf8-b457-13849a254209 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:19.078431 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6adc9-ab84-4cf8-b457-13849a254209 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:19.088125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6adc9-ab84-4cf8-b457-13849a254209 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:19.089515 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6adc9-ab84-4cf8-b457-13849a254209 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:19.099156 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1312/2601] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:19 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:19.174563 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726daffe-4969-4103-8ec2-d91e61a6532d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:19.185768 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726daffe-4969-4103-8ec2-d91e61a6532d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:19.192440 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bda7c4-2a1b-49c4-827e-3861a545fa56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:19.193097 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bda7c4-2a1b-49c4-827e-3861a545fa56 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:19.193455 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bda7c4-2a1b-49c4-827e-3861a545fa56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:19.193666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bda7c4-2a1b-49c4-827e-3861a545fa56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:19.255672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:19.258276 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bda7c4-2a1b-49c4-827e-3861a545fa56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:19.260113 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bda7c4-2a1b-49c4-827e-3861a545fa56 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:19.270325 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1290/2602] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:19 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:21.351236 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6adc9-ab84-4cf8-b457-13849a254209 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:21.419809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6adc9-ab84-4cf8-b457-13849a254209 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:21.427268 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8ef1a-37ac-4acb-975f-7518aafdd435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:21.428318 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8ef1a-37ac-4acb-975f-7518aafdd435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:21.428635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8ef1a-37ac-4acb-975f-7518aafdd435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:21.428955 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8ef1a-37ac-4acb-975f-7518aafdd435 admin admin] PATH_INFO: `/v3/users/c31ccf9915c94bd59dcecd4a310e15a9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:21.430240 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8ef1a-37ac-4acb-975f-7518aafdd435 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c31ccf9915c94bd59dcecd4a310e15a9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:21.432611 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8ef1a-37ac-4acb-975f-7518aafdd435 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:21.484062 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8bf8ef1a-37ac-4acb-975f-7518aafdd435 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c31ccf9915c94bd59dcecd4a310e15a9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:21.484751 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8bf8ef1a-37ac-4acb-975f-7518aafdd435 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c31ccf9915c94bd59dcecd4a310e15a9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:21.485386 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8bf8ef1a-37ac-4acb-975f-7518aafdd435 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c31ccf9915c94bd59dcecd4a310e15a9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:21.500093 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8bf8ef1a-37ac-4acb-975f-7518aafdd435 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c31ccf9915c94bd59dcecd4a310e15a9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:21.507964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8bf8ef1a-37ac-4acb-975f-7518aafdd435 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c31ccf9915c94bd59dcecd4a310e15a9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:21.523342 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1313/2603] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:21 2022] DELETE /identity/v3/users/c31ccf9915c94bd59dcecd4a310e15a9 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:21.528154 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8ad0a-33a0-4d0f-b142-3ee621ccaa73 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:21.528655 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8ad0a-33a0-4d0f-b142-3ee621ccaa73 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:21.528750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8ad0a-33a0-4d0f-b142-3ee621ccaa73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:21.551131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-77e8ad0a-33a0-4d0f-b142-3ee621ccaa73 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:21.553452 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-77e8ad0a-33a0-4d0f-b142-3ee621ccaa73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:21.623842 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1291/2604] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:21.634064 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8ef1a-37ac-4acb-975f-7518aafdd435 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:21.635618 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8bf8ef1a-37ac-4acb-975f-7518aafdd435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:21.693808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8ef1a-37ac-4acb-975f-7518aafdd435 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:21.698132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e393775-fc95-4a9a-bab5-9bb8b2b8b072 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:21.698777 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e393775-fc95-4a9a-bab5-9bb8b2b8b072 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:21.699060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e393775-fc95-4a9a-bab5-9bb8b2b8b072 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:21.699335 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e393775-fc95-4a9a-bab5-9bb8b2b8b072 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:21.708979 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e393775-fc95-4a9a-bab5-9bb8b2b8b072 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:21.710431 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e393775-fc95-4a9a-bab5-9bb8b2b8b072 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:21.720300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1314/2605] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:21.994144 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8ad0a-33a0-4d0f-b142-3ee621ccaa73 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:21.997937 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-77e8ad0a-33a0-4d0f-b142-3ee621ccaa73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:22.016837 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8ad0a-33a0-4d0f-b142-3ee621ccaa73 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:22.022068 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8377f43-e015-4722-a32a-d73a2b81126e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:22.022988 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8377f43-e015-4722-a32a-d73a2b81126e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:22.023215 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8377f43-e015-4722-a32a-d73a2b81126e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:22.023481 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8377f43-e015-4722-a32a-d73a2b81126e admin admin] PATH_INFO: `/v3/projects/f74199d04bc04885855dbbf7b8457d1e` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:22.024835 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8377f43-e015-4722-a32a-d73a2b81126e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f74199d04bc04885855dbbf7b8457d1e)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:22.026799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8377f43-e015-4722-a32a-d73a2b81126e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:22.078631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d8377f43-e015-4722-a32a-d73a2b81126e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f74199d04bc04885855dbbf7b8457d1e', 'request_id': 'req-d8377f43-e015-4722-a32a-d73a2b81126e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:22.088497 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d8377f43-e015-4722-a32a-d73a2b81126e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f74199d04bc04885855dbbf7b8457d1e', 'request_id': 'req-d8377f43-e015-4722-a32a-d73a2b81126e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:22.094887 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d8377f43-e015-4722-a32a-d73a2b81126e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f74199d04bc04885855dbbf7b8457d1e', 'request_id': 'req-d8377f43-e015-4722-a32a-d73a2b81126e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:22.106491 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d8377f43-e015-4722-a32a-d73a2b81126e admin admin] The token cache is being invalidate because project f74199d04bc04885855dbbf7b8457d1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:22.106787 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d8377f43-e015-4722-a32a-d73a2b81126e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:22.107631 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d8377f43-e015-4722-a32a-d73a2b81126e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:22.109172 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1292/2606] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:21 2022] DELETE /identity/v3/projects/f74199d04bc04885855dbbf7b8457d1e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:22.113221 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e393775-fc95-4a9a-bab5-9bb8b2b8b072 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:22.123823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e393775-fc95-4a9a-bab5-9bb8b2b8b072 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:22.129762 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:22.196965 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c00618-43b3-4365-9114-eaba6377f1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:22.197872 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c00618-43b3-4365-9114-eaba6377f1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:22.198150 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c00618-43b3-4365-9114-eaba6377f1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:22.198421 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c00618-43b3-4365-9114-eaba6377f1e3 admin admin] PATH_INFO: `/v3/users/40c66fa811414f5cbe1554958abadd4c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:22.199669 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c00618-43b3-4365-9114-eaba6377f1e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40c66fa811414f5cbe1554958abadd4c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:22.201599 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c00618-43b3-4365-9114-eaba6377f1e3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:22.254635 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-06c00618-43b3-4365-9114-eaba6377f1e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40c66fa811414f5cbe1554958abadd4c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:22.255300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-06c00618-43b3-4365-9114-eaba6377f1e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40c66fa811414f5cbe1554958abadd4c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:22.255938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-06c00618-43b3-4365-9114-eaba6377f1e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40c66fa811414f5cbe1554958abadd4c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:22.268323 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-06c00618-43b3-4365-9114-eaba6377f1e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40c66fa811414f5cbe1554958abadd4c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:22.275374 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-06c00618-43b3-4365-9114-eaba6377f1e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40c66fa811414f5cbe1554958abadd4c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:22.290503 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1315/2607] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:22 2022] DELETE /identity/v3/users/40c66fa811414f5cbe1554958abadd4c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:22.564807 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8377f43-e015-4722-a32a-d73a2b81126e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:22.566504 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d8377f43-e015-4722-a32a-d73a2b81126e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:22.580163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8377f43-e015-4722-a32a-d73a2b81126e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:22.638228 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01639af-79cf-4194-9d8a-01c7ca5fdc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:22.639025 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01639af-79cf-4194-9d8a-01c7ca5fdc40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:22.639307 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01639af-79cf-4194-9d8a-01c7ca5fdc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:22.639583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01639af-79cf-4194-9d8a-01c7ca5fdc40 admin admin] PATH_INFO: `/v3/projects/bcb1c17035d04e1887799ae3078f3345` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:22.640806 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01639af-79cf-4194-9d8a-01c7ca5fdc40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcb1c17035d04e1887799ae3078f3345)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:22.642772 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01639af-79cf-4194-9d8a-01c7ca5fdc40 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:22.665165 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:22.690449 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a01639af-79cf-4194-9d8a-01c7ca5fdc40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcb1c17035d04e1887799ae3078f3345', 'request_id': 'req-a01639af-79cf-4194-9d8a-01c7ca5fdc40'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:22.698546 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a01639af-79cf-4194-9d8a-01c7ca5fdc40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcb1c17035d04e1887799ae3078f3345', 'request_id': 'req-a01639af-79cf-4194-9d8a-01c7ca5fdc40'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:22.704043 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a01639af-79cf-4194-9d8a-01c7ca5fdc40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcb1c17035d04e1887799ae3078f3345', 'request_id': 'req-a01639af-79cf-4194-9d8a-01c7ca5fdc40'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:22.715348 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a01639af-79cf-4194-9d8a-01c7ca5fdc40 admin admin] The token cache is being invalidate because project bcb1c17035d04e1887799ae3078f3345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:22.715659 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a01639af-79cf-4194-9d8a-01c7ca5fdc40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:22.716287 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a01639af-79cf-4194-9d8a-01c7ca5fdc40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:22.717891 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1293/2608] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:22 2022] DELETE /identity/v3/projects/bcb1c17035d04e1887799ae3078f3345 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:22.723001 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83628db-e40a-4aa1-a3ea-ecf6fa4553b0 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:22.723483 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83628db-e40a-4aa1-a3ea-ecf6fa4553b0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:22.723597 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83628db-e40a-4aa1-a3ea-ecf6fa4553b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:22.742069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c83628db-e40a-4aa1-a3ea-ecf6fa4553b0 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:22.743281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c83628db-e40a-4aa1-a3ea-ecf6fa4553b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:22.809992 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1316/2609] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:22.817957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6899cf-603e-49f3-977d-5c7255170c37 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:22.818243 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6899cf-603e-49f3-977d-5c7255170c37 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:22.818520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6899cf-603e-49f3-977d-5c7255170c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:22.838488 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-aa6899cf-603e-49f3-977d-5c7255170c37 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:22.839756 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-aa6899cf-603e-49f3-977d-5c7255170c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:22.848263 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1294/2610] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:22.852863 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83628db-e40a-4aa1-a3ea-ecf6fa4553b0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:22.862973 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83628db-e40a-4aa1-a3ea-ecf6fa4553b0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:22.868073 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67187b1-cc39-450a-8e10-92f9583f9e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:22.868734 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67187b1-cc39-450a-8e10-92f9583f9e30 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:22.869013 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67187b1-cc39-450a-8e10-92f9583f9e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:22.869293 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67187b1-cc39-450a-8e10-92f9583f9e30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:22.870106 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67187b1-cc39-450a-8e10-92f9583f9e30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:23:22.870418 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67187b1-cc39-450a-8e10-92f9583f9e30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:22.872076 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67187b1-cc39-450a-8e10-92f9583f9e30 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:22.884509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1317/2611] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:23:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:23:22.889185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6899cf-603e-49f3-977d-5c7255170c37 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:22.898913 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6899cf-603e-49f3-977d-5c7255170c37 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:22.904932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c17ab-cc3a-4946-b8ea-9dc3f6f1251b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:22.905581 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c17ab-cc3a-4946-b8ea-9dc3f6f1251b admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:22.905861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c17ab-cc3a-4946-b8ea-9dc3f6f1251b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:22.906135 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c17ab-cc3a-4946-b8ea-9dc3f6f1251b admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:22.906801 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c17ab-cc3a-4946-b8ea-9dc3f6f1251b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:22.908745 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c17ab-cc3a-4946-b8ea-9dc3f6f1251b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:22.935956 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1295/2612] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:22 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:22.940165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67187b1-cc39-450a-8e10-92f9583f9e30 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:22.950770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67187b1-cc39-450a-8e10-92f9583f9e30 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:22.962467 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:23.010166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75854a7-c46d-45cd-a4cf-92658cc0a0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:23.010847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75854a7-c46d-45cd-a4cf-92658cc0a0b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:23.011131 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75854a7-c46d-45cd-a4cf-92658cc0a0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:23.011407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75854a7-c46d-45cd-a4cf-92658cc0a0b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:23.012130 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75854a7-c46d-45cd-a4cf-92658cc0a0b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:23.014079 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75854a7-c46d-45cd-a4cf-92658cc0a0b7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:23.036777 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1318/2613] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:22 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:23.042643 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c17ab-cc3a-4946-b8ea-9dc3f6f1251b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:23.058559 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c17ab-cc3a-4946-b8ea-9dc3f6f1251b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:23.063715 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ab07a-dda1-4923-a29a-9cf01718cfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:23.064377 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ab07a-dda1-4923-a29a-9cf01718cfc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:23.064660 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ab07a-dda1-4923-a29a-9cf01718cfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:23.064940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ab07a-dda1-4923-a29a-9cf01718cfc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:23.065664 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ab07a-dda1-4923-a29a-9cf01718cfc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:23.067267 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ab07a-dda1-4923-a29a-9cf01718cfc0 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:23.078293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1296/2614] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:23.082176 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75854a7-c46d-45cd-a4cf-92658cc0a0b7 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:23.092982 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75854a7-c46d-45cd-a4cf-92658cc0a0b7 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:23.098246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8190a-5ef5-4642-bcd3-bb158f1dd329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:23.098917 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8190a-5ef5-4642-bcd3-bb158f1dd329 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:23.099200 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8190a-5ef5-4642-bcd3-bb158f1dd329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:23.099480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8190a-5ef5-4642-bcd3-bb158f1dd329 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:23.100279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8190a-5ef5-4642-bcd3-bb158f1dd329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:23.101892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8190a-5ef5-4642-bcd3-bb158f1dd329 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:23.112282 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1319/2615] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:23.116476 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ab07a-dda1-4923-a29a-9cf01718cfc0 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:23.126922 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ab07a-dda1-4923-a29a-9cf01718cfc0 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:23.133144 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8ef4c-49c2-4266-a564-45493473cd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:23.134027 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8ef4c-49c2-4266-a564-45493473cd95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:23.134307 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8ef4c-49c2-4266-a564-45493473cd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:23.134576 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8ef4c-49c2-4266-a564-45493473cd95 admin admin] PATH_INFO: `/v3/projects/61525140325a4b14aa10df2b64efa5cd/users/19e54ac48c1943b0999baeaad67a0e11/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:23.149896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8ef4c-49c2-4266-a564-45493473cd95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61525140325a4b14aa10df2b64efa5cd, user_id=19e54ac48c1943b0999baeaad67a0e11, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:23.152570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8ef4c-49c2-4266-a564-45493473cd95 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:23.166750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1297/2616] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:23 2022] PUT /identity/v3/projects/61525140325a4b14aa10df2b64efa5cd/users/19e54ac48c1943b0999baeaad67a0e11/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:23.174543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919451d-4d56-4071-afe9-757e1862f6a1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:23.174897 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919451d-4d56-4071-afe9-757e1862f6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:23.175233 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919451d-4d56-4071-afe9-757e1862f6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:23.194000 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2919451d-4d56-4071-afe9-757e1862f6a1 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:23.195289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2919451d-4d56-4071-afe9-757e1862f6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:23.260634 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1320/2617] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:23.270952 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8ef4c-49c2-4266-a564-45493473cd95 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:23.273136 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8bf8ef4c-49c2-4266-a564-45493473cd95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:23.310207 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919451d-4d56-4071-afe9-757e1862f6a1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:23.311878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2919451d-4d56-4071-afe9-757e1862f6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:23.324330 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919451d-4d56-4071-afe9-757e1862f6a1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:23.329097 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec080b5-affc-4545-8e41-c87a07e83b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:23.330101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec080b5-affc-4545-8e41-c87a07e83b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:23.330252 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec080b5-affc-4545-8e41-c87a07e83b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:23.330520 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec080b5-affc-4545-8e41-c87a07e83b1a admin admin] PATH_INFO: `/v3/users/c899129975324d4798277f09b755792e` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:23.331762 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec080b5-affc-4545-8e41-c87a07e83b1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c899129975324d4798277f09b755792e)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:23.333740 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec080b5-affc-4545-8e41-c87a07e83b1a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:23.335696 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8ef4c-49c2-4266-a564-45493473cd95 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:23.339454 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d5255-b9a6-4a7a-bc7a-8ace8dc86ba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:23.340123 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d5255-b9a6-4a7a-bc7a-8ace8dc86ba4 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:23.340423 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d5255-b9a6-4a7a-bc7a-8ace8dc86ba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:23.340673 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d5255-b9a6-4a7a-bc7a-8ace8dc86ba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:23.350455 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d5255-b9a6-4a7a-bc7a-8ace8dc86ba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:23.352161 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d5255-b9a6-4a7a-bc7a-8ace8dc86ba4 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:23.363037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1298/2618] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:23.381776 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9ec080b5-affc-4545-8e41-c87a07e83b1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c899129975324d4798277f09b755792e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:23.382441 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9ec080b5-affc-4545-8e41-c87a07e83b1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c899129975324d4798277f09b755792e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:23.383062 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9ec080b5-affc-4545-8e41-c87a07e83b1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c899129975324d4798277f09b755792e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:23.395340 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9ec080b5-affc-4545-8e41-c87a07e83b1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c899129975324d4798277f09b755792e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:23.402162 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9ec080b5-affc-4545-8e41-c87a07e83b1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c899129975324d4798277f09b755792e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:23.414642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1321/2619] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:23 2022] DELETE /identity/v3/users/c899129975324d4798277f09b755792e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:23.419256 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bda798-591d-45b7-a0e2-08da795646f9 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:23.419590 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bda798-591d-45b7-a0e2-08da795646f9 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:23.419897 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bda798-591d-45b7-a0e2-08da795646f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:23.437226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e9bda798-591d-45b7-a0e2-08da795646f9 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:23.438451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e9bda798-591d-45b7-a0e2-08da795646f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:23.484994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:23.502639 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1299/2620] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:23.514068 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec080b5-affc-4545-8e41-c87a07e83b1a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:23.515876 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9ec080b5-affc-4545-8e41-c87a07e83b1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:23.526387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:23.580782 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec080b5-affc-4545-8e41-c87a07e83b1a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:23.584515 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f51ae8-0a49-4c8f-bc18-01448702e555 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:23.585155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f51ae8-0a49-4c8f-bc18-01448702e555 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:23.585437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f51ae8-0a49-4c8f-bc18-01448702e555 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:23.585710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f51ae8-0a49-4c8f-bc18-01448702e555 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:23.595352 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f51ae8-0a49-4c8f-bc18-01448702e555 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:23.596851 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f51ae8-0a49-4c8f-bc18-01448702e555 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:23.606813 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1322/2621] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:23.885309 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bda798-591d-45b7-a0e2-08da795646f9 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:23.887471 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e9bda798-591d-45b7-a0e2-08da795646f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:23.900512 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bda798-591d-45b7-a0e2-08da795646f9 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:23.906109 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029dc91-8d7e-4126-963b-fc037eedf190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:23.907432 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029dc91-8d7e-4126-963b-fc037eedf190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:23.907821 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029dc91-8d7e-4126-963b-fc037eedf190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:23.908591 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029dc91-8d7e-4126-963b-fc037eedf190 admin admin] PATH_INFO: `/v3/projects/13fb1c6a91e64b42aa6677c5d0754b05` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:23.909481 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8029dc91-8d7e-4126-963b-fc037eedf190 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13fb1c6a91e64b42aa6677c5d0754b05)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:23.911755 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8029dc91-8d7e-4126-963b-fc037eedf190 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:23.961267 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8029dc91-8d7e-4126-963b-fc037eedf190 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13fb1c6a91e64b42aa6677c5d0754b05', 'request_id': 'req-8029dc91-8d7e-4126-963b-fc037eedf190'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:23.969660 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8029dc91-8d7e-4126-963b-fc037eedf190 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13fb1c6a91e64b42aa6677c5d0754b05', 'request_id': 'req-8029dc91-8d7e-4126-963b-fc037eedf190'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:23.979090 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8029dc91-8d7e-4126-963b-fc037eedf190 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13fb1c6a91e64b42aa6677c5d0754b05', 'request_id': 'req-8029dc91-8d7e-4126-963b-fc037eedf190'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:23.992730 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8029dc91-8d7e-4126-963b-fc037eedf190 admin admin] The token cache is being invalidate because project 13fb1c6a91e64b42aa6677c5d0754b05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:23.993020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8029dc91-8d7e-4126-963b-fc037eedf190 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:23.993682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8029dc91-8d7e-4126-963b-fc037eedf190 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:23.995537 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1300/2622] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:23 2022] DELETE /identity/v3/projects/13fb1c6a91e64b42aa6677c5d0754b05 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:23.999500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f51ae8-0a49-4c8f-bc18-01448702e555 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:24.019496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f51ae8-0a49-4c8f-bc18-01448702e555 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:24.083168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e558c-4257-4460-b7cd-057325d99b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:24.083911 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e558c-4257-4460-b7cd-057325d99b55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:24.084194 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e558c-4257-4460-b7cd-057325d99b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:24.084468 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e558c-4257-4460-b7cd-057325d99b55 admin admin] PATH_INFO: `/v3/users/027ea69f071743c5ac36c19219ce44a2` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:24.089547 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e558c-4257-4460-b7cd-057325d99b55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=027ea69f071743c5ac36c19219ce44a2)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:24.091650 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e558c-4257-4460-b7cd-057325d99b55 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:24.151800 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b99e558c-4257-4460-b7cd-057325d99b55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '027ea69f071743c5ac36c19219ce44a2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:24.152606 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b99e558c-4257-4460-b7cd-057325d99b55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '027ea69f071743c5ac36c19219ce44a2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:24.153202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b99e558c-4257-4460-b7cd-057325d99b55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '027ea69f071743c5ac36c19219ce44a2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:24.169125 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b99e558c-4257-4460-b7cd-057325d99b55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '027ea69f071743c5ac36c19219ce44a2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:24.180710 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b99e558c-4257-4460-b7cd-057325d99b55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '027ea69f071743c5ac36c19219ce44a2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:24.199034 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1323/2623] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:23 2022] DELETE /identity/v3/users/027ea69f071743c5ac36c19219ce44a2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:24.510832 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029dc91-8d7e-4126-963b-fc037eedf190 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:24.513222 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8029dc91-8d7e-4126-963b-fc037eedf190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:24.528088 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029dc91-8d7e-4126-963b-fc037eedf190 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:24.604127 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84b301-85a3-408c-b6d4-bdc95b8aca92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:24.605302 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84b301-85a3-408c-b6d4-bdc95b8aca92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:24.605649 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84b301-85a3-408c-b6d4-bdc95b8aca92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:24.605992 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84b301-85a3-408c-b6d4-bdc95b8aca92 admin admin] PATH_INFO: `/v3/projects/e0af58b5da964b2aaa2bacffda5940b6` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:24.607453 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc84b301-85a3-408c-b6d4-bdc95b8aca92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0af58b5da964b2aaa2bacffda5940b6)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:24.609760 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc84b301-85a3-408c-b6d4-bdc95b8aca92 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:24.667733 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bc84b301-85a3-408c-b6d4-bdc95b8aca92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0af58b5da964b2aaa2bacffda5940b6', 'request_id': 'req-bc84b301-85a3-408c-b6d4-bdc95b8aca92'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:24.677026 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bc84b301-85a3-408c-b6d4-bdc95b8aca92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0af58b5da964b2aaa2bacffda5940b6', 'request_id': 'req-bc84b301-85a3-408c-b6d4-bdc95b8aca92'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:24.685952 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bc84b301-85a3-408c-b6d4-bdc95b8aca92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0af58b5da964b2aaa2bacffda5940b6', 'request_id': 'req-bc84b301-85a3-408c-b6d4-bdc95b8aca92'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:24.700829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bc84b301-85a3-408c-b6d4-bdc95b8aca92 admin admin] The token cache is being invalidate because project e0af58b5da964b2aaa2bacffda5940b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:24.701348 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bc84b301-85a3-408c-b6d4-bdc95b8aca92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:24.707978 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bc84b301-85a3-408c-b6d4-bdc95b8aca92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:24.709900 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1301/2624] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:24 2022] DELETE /identity/v3/projects/e0af58b5da964b2aaa2bacffda5940b6 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:24.715029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364a3d0-3bed-4dfb-9592-42e80f9f5c25 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:24.715315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364a3d0-3bed-4dfb-9592-42e80f9f5c25 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:24.715591 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364a3d0-3bed-4dfb-9592-42e80f9f5c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:24.735578 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e364a3d0-3bed-4dfb-9592-42e80f9f5c25 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:24.736838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e364a3d0-3bed-4dfb-9592-42e80f9f5c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:24.765275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:24.807129 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1324/2625] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:24.815077 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f3a2b-f5af-459a-8049-d1a86f48fb31 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:24.815426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f3a2b-f5af-459a-8049-d1a86f48fb31 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:24.815805 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f3a2b-f5af-459a-8049-d1a86f48fb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:24.823342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:24.847541 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-883f3a2b-f5af-459a-8049-d1a86f48fb31 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:24.849005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-883f3a2b-f5af-459a-8049-d1a86f48fb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:24.858325 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1302/2626] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:24.865140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364a3d0-3bed-4dfb-9592-42e80f9f5c25 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:24.899232 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364a3d0-3bed-4dfb-9592-42e80f9f5c25 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:24.906782 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7949c-a069-4822-9057-1bd61e361cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:24.907473 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7949c-a069-4822-9057-1bd61e361cbe admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:24.907816 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7949c-a069-4822-9057-1bd61e361cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:24.908092 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7949c-a069-4822-9057-1bd61e361cbe admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:24.908828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7949c-a069-4822-9057-1bd61e361cbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:23:24.909135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7949c-a069-4822-9057-1bd61e361cbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:24.910864 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7949c-a069-4822-9057-1bd61e361cbe admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:24.924315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1325/2627] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:23:24.927522 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f3a2b-f5af-459a-8049-d1a86f48fb31 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:24.948020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f3a2b-f5af-459a-8049-d1a86f48fb31 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:24.960991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16605c-6adb-4863-ba9e-328416caf9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:24.961830 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16605c-6adb-4863-ba9e-328416caf9e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:24.962118 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16605c-6adb-4863-ba9e-328416caf9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:24.962391 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16605c-6adb-4863-ba9e-328416caf9e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:24.963156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16605c-6adb-4863-ba9e-328416caf9e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:24.965346 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16605c-6adb-4863-ba9e-328416caf9e4 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:25.000965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1303/2628] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:24 2022] POST /identity/v3/projects => generated 379 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:25.005410 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7949c-a069-4822-9057-1bd61e361cbe admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:25.021023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7949c-a069-4822-9057-1bd61e361cbe admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:25.080455 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18167e27-df08-481a-8a1c-72d61047248a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:25.081136 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18167e27-df08-481a-8a1c-72d61047248a admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:25.081419 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18167e27-df08-481a-8a1c-72d61047248a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:25.081700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18167e27-df08-481a-8a1c-72d61047248a admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:25.082389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18167e27-df08-481a-8a1c-72d61047248a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:25.084374 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18167e27-df08-481a-8a1c-72d61047248a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:25.104706 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1326/2629] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:25 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:25.108752 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16605c-6adb-4863-ba9e-328416caf9e4 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:25.119365 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16605c-6adb-4863-ba9e-328416caf9e4 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:25.124753 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246cc72c-b7db-450f-9497-cdcd45d22551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:25.125430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cc72c-b7db-450f-9497-cdcd45d22551 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:25.125742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cc72c-b7db-450f-9497-cdcd45d22551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:25.126021 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cc72c-b7db-450f-9497-cdcd45d22551 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:25.126906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246cc72c-b7db-450f-9497-cdcd45d22551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:25.128882 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246cc72c-b7db-450f-9497-cdcd45d22551 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:25.139953 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1304/2630] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:25.143936 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18167e27-df08-481a-8a1c-72d61047248a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:25.154199 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18167e27-df08-481a-8a1c-72d61047248a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:25.158887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6c135-1cbf-4c76-b217-d3658b27c95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:25.159554 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6c135-1cbf-4c76-b217-d3658b27c95b admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:25.159902 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6c135-1cbf-4c76-b217-d3658b27c95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:25.160179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6c135-1cbf-4c76-b217-d3658b27c95b admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:25.160926 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6c135-1cbf-4c76-b217-d3658b27c95b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:25.162545 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6c135-1cbf-4c76-b217-d3658b27c95b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:25.177455 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1327/2631] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:25.178165 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246cc72c-b7db-450f-9497-cdcd45d22551 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:25.194465 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246cc72c-b7db-450f-9497-cdcd45d22551 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:25.199576 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9daec7-719c-44e5-9ea5-3a009ac63946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:25.200525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9daec7-719c-44e5-9ea5-3a009ac63946 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:25.200806 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9daec7-719c-44e5-9ea5-3a009ac63946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:25.201084 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9daec7-719c-44e5-9ea5-3a009ac63946 admin admin] PATH_INFO: `/v3/projects/31c6d958c3e8429083db0f0f9f54decc/users/11f3608a85414efc88b8d7067af65fbb/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:25.216405 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9daec7-719c-44e5-9ea5-3a009ac63946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31c6d958c3e8429083db0f0f9f54decc, user_id=11f3608a85414efc88b8d7067af65fbb, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:25.219082 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9daec7-719c-44e5-9ea5-3a009ac63946 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:25.234958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1305/2632] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:25 2022] PUT /identity/v3/projects/31c6d958c3e8429083db0f0f9f54decc/users/11f3608a85414efc88b8d7067af65fbb/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:25.246355 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0adea-373e-4c92-bba5-247d2cf7e6d1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:25.246622 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0adea-373e-4c92-bba5-247d2cf7e6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:25.246899 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0adea-373e-4c92-bba5-247d2cf7e6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:25.273989 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b3f0adea-373e-4c92-bba5-247d2cf7e6d1 None None] MFA Rules not processed for user `11f3608a85414efc88b8d7067af65fbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:25.275192 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b3f0adea-373e-4c92-bba5-247d2cf7e6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:25.341013 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1328/2633] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:25.345595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9daec7-719c-44e5-9ea5-3a009ac63946 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:25.413137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9daec7-719c-44e5-9ea5-3a009ac63946 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:25.418775 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e61e3-d842-4349-928c-3da758c1a960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:25.419555 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e61e3-d842-4349-928c-3da758c1a960 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:25.419916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e61e3-d842-4349-928c-3da758c1a960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:25.420259 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e61e3-d842-4349-928c-3da758c1a960 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:25.421037 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e61e3-d842-4349-928c-3da758c1a960 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:25.423031 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e61e3-d842-4349-928c-3da758c1a960 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:25.453769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1306/2634] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:25 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:25.457508 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0adea-373e-4c92-bba5-247d2cf7e6d1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:25.468891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0adea-373e-4c92-bba5-247d2cf7e6d1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:25.490791 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:25.534235 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d548a-fc7b-4e5c-8edb-f9e63e9ccafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:25.534938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d548a-fc7b-4e5c-8edb-f9e63e9ccafa admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:25.535219 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d548a-fc7b-4e5c-8edb-f9e63e9ccafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:25.535497 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d548a-fc7b-4e5c-8edb-f9e63e9ccafa admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:25.536323 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d548a-fc7b-4e5c-8edb-f9e63e9ccafa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:25.538237 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d548a-fc7b-4e5c-8edb-f9e63e9ccafa admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:25.571715 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1329/2635] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:25 2022] POST /identity/v3/users => generated 290 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:25.575442 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e61e3-d842-4349-928c-3da758c1a960 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:25.593683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e61e3-d842-4349-928c-3da758c1a960 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:25.600980 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715a51a-83da-468a-8847-1e91339cd136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:25.601683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715a51a-83da-468a-8847-1e91339cd136 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:25.602689 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715a51a-83da-468a-8847-1e91339cd136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:25.602985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715a51a-83da-468a-8847-1e91339cd136 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:25.603922 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715a51a-83da-468a-8847-1e91339cd136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:25.607001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715a51a-83da-468a-8847-1e91339cd136 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:25.627417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1307/2636] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:23:25.628975 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d548a-fc7b-4e5c-8edb-f9e63e9ccafa admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:25.641642 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d548a-fc7b-4e5c-8edb-f9e63e9ccafa admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:25.647112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef749fc7-eab2-4351-825b-bd5a821e8fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:25.648077 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef749fc7-eab2-4351-825b-bd5a821e8fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:25.648427 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef749fc7-eab2-4351-825b-bd5a821e8fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:25.648814 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef749fc7-eab2-4351-825b-bd5a821e8fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:25.650949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef749fc7-eab2-4351-825b-bd5a821e8fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:25.652166 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef749fc7-eab2-4351-825b-bd5a821e8fb8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:25.665942 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1330/2637] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:25.670852 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715a51a-83da-468a-8847-1e91339cd136 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:25.683258 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715a51a-83da-468a-8847-1e91339cd136 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:25.689974 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94997547-ccbf-48be-b545-ef87e312d3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:25.691294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94997547-ccbf-48be-b545-ef87e312d3aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:25.691688 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94997547-ccbf-48be-b545-ef87e312d3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:25.692059 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94997547-ccbf-48be-b545-ef87e312d3aa admin admin] PATH_INFO: `/v3/projects/bcb78fcf0b604a068b358c61f8e6b54c/users/5ccc24707b2942f8abbb655154d9c8a2/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:25.711589 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94997547-ccbf-48be-b545-ef87e312d3aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcb78fcf0b604a068b358c61f8e6b54c, user_id=5ccc24707b2942f8abbb655154d9c8a2, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:25.714438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94997547-ccbf-48be-b545-ef87e312d3aa admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:25.733720 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1308/2638] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:25 2022] PUT /identity/v3/projects/bcb78fcf0b604a068b358c61f8e6b54c/users/5ccc24707b2942f8abbb655154d9c8a2/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:25.744388 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a96b91-61fc-45f3-be84-ed33bad9d84f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:25.744688 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a96b91-61fc-45f3-be84-ed33bad9d84f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:25.745072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a96b91-61fc-45f3-be84-ed33bad9d84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:25.779110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a9a96b91-61fc-45f3-be84-ed33bad9d84f None None] MFA Rules not processed for user `5ccc24707b2942f8abbb655154d9c8a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:25.779348 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a9a96b91-61fc-45f3-be84-ed33bad9d84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:25.870664 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1331/2639] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:25.877162 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94997547-ccbf-48be-b545-ef87e312d3aa admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:25.951920 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:25.982497 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94997547-ccbf-48be-b545-ef87e312d3aa admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:26.000815 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63326b-c8bf-4567-9968-91f8f0c37382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:26.001530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63326b-c8bf-4567-9968-91f8f0c37382 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:26.001812 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63326b-c8bf-4567-9968-91f8f0c37382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:26.002089 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63326b-c8bf-4567-9968-91f8f0c37382 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:26.002794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63326b-c8bf-4567-9968-91f8f0c37382 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:26.005099 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63326b-c8bf-4567-9968-91f8f0c37382 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:26.042158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1309/2640] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:25 2022] POST /identity/v3/projects => generated 375 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:26.046403 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a96b91-61fc-45f3-be84-ed33bad9d84f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:26.061393 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a96b91-61fc-45f3-be84-ed33bad9d84f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:26.149811 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8d401-9e1f-41bb-9ceb-4ca880c00fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:26.150800 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8d401-9e1f-41bb-9ceb-4ca880c00fab admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:26.151191 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8d401-9e1f-41bb-9ceb-4ca880c00fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:26.151569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8d401-9e1f-41bb-9ceb-4ca880c00fab admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:26.152532 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8d401-9e1f-41bb-9ceb-4ca880c00fab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:26.155631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8d401-9e1f-41bb-9ceb-4ca880c00fab admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:26.183447 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1332/2641] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:26 2022] POST /identity/v3/users => generated 294 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:26.184875 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63326b-c8bf-4567-9968-91f8f0c37382 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:26.199080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63326b-c8bf-4567-9968-91f8f0c37382 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:26.204693 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ddf78-0460-45d2-a675-e6ad57fbf7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:26.205059 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ddf78-0460-45d2-a675-e6ad57fbf7b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:26.205059 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ddf78-0460-45d2-a675-e6ad57fbf7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:26.205353 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ddf78-0460-45d2-a675-e6ad57fbf7b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:26.206297 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ddf78-0460-45d2-a675-e6ad57fbf7b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:26.208096 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ddf78-0460-45d2-a675-e6ad57fbf7b7 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:26.223525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1310/2642] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:26.224990 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8d401-9e1f-41bb-9ceb-4ca880c00fab admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:26.239929 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8d401-9e1f-41bb-9ceb-4ca880c00fab admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:26.245980 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac39430-5f01-49e7-8a2b-40e2e0abadeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:26.247945 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac39430-5f01-49e7-8a2b-40e2e0abadeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:26.248438 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac39430-5f01-49e7-8a2b-40e2e0abadeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:26.248893 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac39430-5f01-49e7-8a2b-40e2e0abadeb admin admin] PATH_INFO: `/v3/projects/fae7d52fd1ea4abd84d3766f147b362d/users/7d92d807159a43fb94ee8967fca7facb/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:26.275656 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac39430-5f01-49e7-8a2b-40e2e0abadeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fae7d52fd1ea4abd84d3766f147b362d, user_id=7d92d807159a43fb94ee8967fca7facb, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:26.280183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac39430-5f01-49e7-8a2b-40e2e0abadeb admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:26.291850 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:26.305570 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1333/2643] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:26 2022] PUT /identity/v3/projects/fae7d52fd1ea4abd84d3766f147b362d/users/7d92d807159a43fb94ee8967fca7facb/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:26.307513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ddf78-0460-45d2-a675-e6ad57fbf7b7 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:26.318444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ddf78-0460-45d2-a675-e6ad57fbf7b7 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:26.389181 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3046c-3d5c-4771-96e1-14abaaeee571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:26.390102 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3046c-3d5c-4771-96e1-14abaaeee571 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:26.390417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3046c-3d5c-4771-96e1-14abaaeee571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:26.390728 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3046c-3d5c-4771-96e1-14abaaeee571 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:26.391820 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3046c-3d5c-4771-96e1-14abaaeee571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:26.393474 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3046c-3d5c-4771-96e1-14abaaeee571 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:26.405931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1311/2644] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:26 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:26.410129 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac39430-5f01-49e7-8a2b-40e2e0abadeb admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:26.421657 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac39430-5f01-49e7-8a2b-40e2e0abadeb admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:26.430709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ff652-4c61-4cec-a40e-5caaeb35e846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:26.431883 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ff652-4c61-4cec-a40e-5caaeb35e846 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:26.432528 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ff652-4c61-4cec-a40e-5caaeb35e846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:26.432810 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ff652-4c61-4cec-a40e-5caaeb35e846 admin admin] PATH_INFO: `/v3/projects/fae7d52fd1ea4abd84d3766f147b362d/users/7d92d807159a43fb94ee8967fca7facb/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:26.435004 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ff652-4c61-4cec-a40e-5caaeb35e846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fae7d52fd1ea4abd84d3766f147b362d, user_id=7d92d807159a43fb94ee8967fca7facb, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:26.437895 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ff652-4c61-4cec-a40e-5caaeb35e846 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:26.451271 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1334/2645] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:26 2022] PUT /identity/v3/projects/fae7d52fd1ea4abd84d3766f147b362d/users/7d92d807159a43fb94ee8967fca7facb/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:26.463498 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78d460-d7ca-4d7c-9c47-96882216e9be None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:26.463937 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78d460-d7ca-4d7c-9c47-96882216e9be None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:26.464153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78d460-d7ca-4d7c-9c47-96882216e9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:26.494434 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ec78d460-d7ca-4d7c-9c47-96882216e9be None None] MFA Rules not processed for user `7d92d807159a43fb94ee8967fca7facb`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:26.495733 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ec78d460-d7ca-4d7c-9c47-96882216e9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:26.588044 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1312/2646] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:26.599702 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ff652-4c61-4cec-a40e-5caaeb35e846 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:26.601956 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ad9ff652-4c61-4cec-a40e-5caaeb35e846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:26.667096 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ff652-4c61-4cec-a40e-5caaeb35e846 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:26.672433 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee50dac-02a6-4bf2-83f2-60463319fb13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:26.673103 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee50dac-02a6-4bf2-83f2-60463319fb13 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:26.673389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee50dac-02a6-4bf2-83f2-60463319fb13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:26.673670 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee50dac-02a6-4bf2-83f2-60463319fb13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:26.685396 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee50dac-02a6-4bf2-83f2-60463319fb13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:26.687023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee50dac-02a6-4bf2-83f2-60463319fb13 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:26.699314 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1335/2647] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:26.749809 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78d460-d7ca-4d7c-9c47-96882216e9be None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:26.752277 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ec78d460-d7ca-4d7c-9c47-96882216e9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:26.771245 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78d460-d7ca-4d7c-9c47-96882216e9be None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:26.779008 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3a585-fbb8-4e90-8ed1-48dfee356c3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:26.783582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3a585-fbb8-4e90-8ed1-48dfee356c3f service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:26.784013 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3a585-fbb8-4e90-8ed1-48dfee356c3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:26.784425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3a585-fbb8-4e90-8ed1-48dfee356c3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:26.840289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:26.877538 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3a585-fbb8-4e90-8ed1-48dfee356c3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:26.886457 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3a585-fbb8-4e90-8ed1-48dfee356c3f service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:26.913836 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1313/2648] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:23:27.690599 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45400639-fd01-4e9b-8cda-3dd95bb21aeb None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:27.691472 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45400639-fd01-4e9b-8cda-3dd95bb21aeb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:27.691782 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45400639-fd01-4e9b-8cda-3dd95bb21aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:27.721228 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-45400639-fd01-4e9b-8cda-3dd95bb21aeb None None] MFA Rules not processed for user `19e54ac48c1943b0999baeaad67a0e11`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:27.722464 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-45400639-fd01-4e9b-8cda-3dd95bb21aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:27.792184 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1336/2649] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:27.796781 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3a585-fbb8-4e90-8ed1-48dfee356c3f service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:27.798220 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-25d3a585-fbb8-4e90-8ed1-48dfee356c3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:27.868047 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3a585-fbb8-4e90-8ed1-48dfee356c3f service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:27.872638 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a222611-77f5-4972-8e43-357892e91927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:27.873314 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a222611-77f5-4972-8e43-357892e91927 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:27.873596 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a222611-77f5-4972-8e43-357892e91927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:27.873874 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a222611-77f5-4972-8e43-357892e91927 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:27.874645 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a222611-77f5-4972-8e43-357892e91927 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:27.876691 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a222611-77f5-4972-8e43-357892e91927 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:27.904063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1314/2650] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:27 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:27.908072 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45400639-fd01-4e9b-8cda-3dd95bb21aeb None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:27.919526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45400639-fd01-4e9b-8cda-3dd95bb21aeb None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:27.977501 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2ea32-e877-450a-a208-54e553306950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:27.978101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2ea32-e877-450a-a208-54e553306950 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:27.978384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2ea32-e877-450a-a208-54e553306950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:27.978771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2ea32-e877-450a-a208-54e553306950 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:27.979507 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2ea32-e877-450a-a208-54e553306950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:27.981476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2ea32-e877-450a-a208-54e553306950 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:28.008090 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1337/2651] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:27 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:28.012261 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a222611-77f5-4972-8e43-357892e91927 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:28.022763 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a222611-77f5-4972-8e43-357892e91927 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:28.027879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0647a0-095a-4ed2-998e-6acf201cc672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:28.028543 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0647a0-095a-4ed2-998e-6acf201cc672 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:28.028825 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0647a0-095a-4ed2-998e-6acf201cc672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:28.029104 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0647a0-095a-4ed2-998e-6acf201cc672 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:28.029829 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0647a0-095a-4ed2-998e-6acf201cc672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:28.031433 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0647a0-095a-4ed2-998e-6acf201cc672 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:28.042311 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1315/2652] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:28.046845 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2ea32-e877-450a-a208-54e553306950 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:28.057650 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2ea32-e877-450a-a208-54e553306950 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:28.062420 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4cd0d-16f5-4590-9a6f-b7550ae1a9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:28.063315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4cd0d-16f5-4590-9a6f-b7550ae1a9bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:28.063627 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4cd0d-16f5-4590-9a6f-b7550ae1a9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:28.063903 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4cd0d-16f5-4590-9a6f-b7550ae1a9bd admin admin] PATH_INFO: `/v3/projects/aa995a6e9663423bb9079d9126f92f0f/users/d7eda7675d7648c3aff7e869a5718358/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:28.066175 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:28.081613 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4cd0d-16f5-4590-9a6f-b7550ae1a9bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa995a6e9663423bb9079d9126f92f0f, user_id=d7eda7675d7648c3aff7e869a5718358, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:28.084325 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4cd0d-16f5-4590-9a6f-b7550ae1a9bd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:28.099960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1338/2653] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:28 2022] PUT /identity/v3/projects/aa995a6e9663423bb9079d9126f92f0f/users/d7eda7675d7648c3aff7e869a5718358/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:28.104507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0647a0-095a-4ed2-998e-6acf201cc672 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:28.120191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0647a0-095a-4ed2-998e-6acf201cc672 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:28.179747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d714d-0db0-454b-a7aa-db99d8664601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:28.180409 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d714d-0db0-454b-a7aa-db99d8664601 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:28.180688 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d714d-0db0-454b-a7aa-db99d8664601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:28.180961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d714d-0db0-454b-a7aa-db99d8664601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:28.181762 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4d714d-0db0-454b-a7aa-db99d8664601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:28.183359 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4d714d-0db0-454b-a7aa-db99d8664601 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:28.196207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1316/2654] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:28.201022 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4cd0d-16f5-4590-9a6f-b7550ae1a9bd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:28.214293 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4cd0d-16f5-4590-9a6f-b7550ae1a9bd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:28.220786 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e045a6-6a30-4f46-b31c-87ae45bede9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:28.222204 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e045a6-6a30-4f46-b31c-87ae45bede9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:28.222485 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e045a6-6a30-4f46-b31c-87ae45bede9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:28.222759 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e045a6-6a30-4f46-b31c-87ae45bede9b admin admin] PATH_INFO: `/v3/projects/aa995a6e9663423bb9079d9126f92f0f/users/d7eda7675d7648c3aff7e869a5718358/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:28.224863 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e045a6-6a30-4f46-b31c-87ae45bede9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa995a6e9663423bb9079d9126f92f0f, user_id=d7eda7675d7648c3aff7e869a5718358, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:28.227585 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e045a6-6a30-4f46-b31c-87ae45bede9b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:28.239056 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1339/2655] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:28 2022] PUT /identity/v3/projects/aa995a6e9663423bb9079d9126f92f0f/users/d7eda7675d7648c3aff7e869a5718358/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:29.978615 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d714d-0db0-454b-a7aa-db99d8664601 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:29.994496 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d714d-0db0-454b-a7aa-db99d8664601 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:30.055597 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:30.071651 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbceaf3-f23b-4656-8bc9-c8529e49a37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:30.074965 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbceaf3-f23b-4656-8bc9-c8529e49a37f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:30.075273 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbceaf3-f23b-4656-8bc9-c8529e49a37f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:30.075547 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbceaf3-f23b-4656-8bc9-c8529e49a37f admin admin] PATH_INFO: `/v3/users/11f3608a85414efc88b8d7067af65fbb` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:30.076844 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbceaf3-f23b-4656-8bc9-c8529e49a37f admin admin] RBAC: Authorizing `identity:delete_user(user_id=11f3608a85414efc88b8d7067af65fbb)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:30.078932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbceaf3-f23b-4656-8bc9-c8529e49a37f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:30.163578 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2fbceaf3-f23b-4656-8bc9-c8529e49a37f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11f3608a85414efc88b8d7067af65fbb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:30.164399 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2fbceaf3-f23b-4656-8bc9-c8529e49a37f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11f3608a85414efc88b8d7067af65fbb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:30.165062 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2fbceaf3-f23b-4656-8bc9-c8529e49a37f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11f3608a85414efc88b8d7067af65fbb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:30.189058 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2fbceaf3-f23b-4656-8bc9-c8529e49a37f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11f3608a85414efc88b8d7067af65fbb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:30.209493 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2fbceaf3-f23b-4656-8bc9-c8529e49a37f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11f3608a85414efc88b8d7067af65fbb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:30.232225 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1317/2656] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:29 2022] DELETE /identity/v3/users/11f3608a85414efc88b8d7067af65fbb => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:30.236992 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857d635-8734-4e8f-8137-86cc79e3cf2f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:30.237532 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857d635-8734-4e8f-8137-86cc79e3cf2f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:30.237532 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857d635-8734-4e8f-8137-86cc79e3cf2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:30.269435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4857d635-8734-4e8f-8137-86cc79e3cf2f None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:30.270690 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4857d635-8734-4e8f-8137-86cc79e3cf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:30.376132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1340/2657] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:30.395632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbceaf3-f23b-4656-8bc9-c8529e49a37f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:30.398890 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2fbceaf3-f23b-4656-8bc9-c8529e49a37f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:30.518870 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbceaf3-f23b-4656-8bc9-c8529e49a37f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:30.525668 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee11f1-b913-4b7b-b846-cab8b723599b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:30.531006 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee11f1-b913-4b7b-b846-cab8b723599b service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:30.531379 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee11f1-b913-4b7b-b846-cab8b723599b service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:30.531697 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee11f1-b913-4b7b-b846-cab8b723599b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:30.549646 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee11f1-b913-4b7b-b846-cab8b723599b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:30.551323 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee11f1-b913-4b7b-b846-cab8b723599b service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:30.563855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1318/2658] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:31.069902 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857d635-8734-4e8f-8137-86cc79e3cf2f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:31.071397 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4857d635-8734-4e8f-8137-86cc79e3cf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:31.096749 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857d635-8734-4e8f-8137-86cc79e3cf2f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:31.106638 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c1a0f-587d-419d-a002-dddbf0df7345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:31.107559 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c1a0f-587d-419d-a002-dddbf0df7345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:31.107882 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c1a0f-587d-419d-a002-dddbf0df7345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:31.108150 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c1a0f-587d-419d-a002-dddbf0df7345 admin admin] PATH_INFO: `/v3/projects/31c6d958c3e8429083db0f0f9f54decc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:31.109554 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5c1a0f-587d-419d-a002-dddbf0df7345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31c6d958c3e8429083db0f0f9f54decc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:31.112345 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5c1a0f-587d-419d-a002-dddbf0df7345 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:31.232647 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bb5c1a0f-587d-419d-a002-dddbf0df7345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31c6d958c3e8429083db0f0f9f54decc', 'request_id': 'req-bb5c1a0f-587d-419d-a002-dddbf0df7345'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:31.242688 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bb5c1a0f-587d-419d-a002-dddbf0df7345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31c6d958c3e8429083db0f0f9f54decc', 'request_id': 'req-bb5c1a0f-587d-419d-a002-dddbf0df7345'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:31.251911 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bb5c1a0f-587d-419d-a002-dddbf0df7345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31c6d958c3e8429083db0f0f9f54decc', 'request_id': 'req-bb5c1a0f-587d-419d-a002-dddbf0df7345'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:31.272307 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bb5c1a0f-587d-419d-a002-dddbf0df7345 admin admin] The token cache is being invalidate because project 31c6d958c3e8429083db0f0f9f54decc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:31.273126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bb5c1a0f-587d-419d-a002-dddbf0df7345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:31.273983 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bb5c1a0f-587d-419d-a002-dddbf0df7345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:31.275937 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1341/2659] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:31 2022] DELETE /identity/v3/projects/31c6d958c3e8429083db0f0f9f54decc => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:31.279778 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee11f1-b913-4b7b-b846-cab8b723599b service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:31.292653 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee11f1-b913-4b7b-b846-cab8b723599b service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:31.364963 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac444c6-eae2-4ac6-826e-3b34e58d320f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:31.366460 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac444c6-eae2-4ac6-826e-3b34e58d320f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:31.366745 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac444c6-eae2-4ac6-826e-3b34e58d320f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:31.367054 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac444c6-eae2-4ac6-826e-3b34e58d320f admin admin] PATH_INFO: `/v3/users/5ccc24707b2942f8abbb655154d9c8a2` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:31.368381 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac444c6-eae2-4ac6-826e-3b34e58d320f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ccc24707b2942f8abbb655154d9c8a2)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:31.371227 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac444c6-eae2-4ac6-826e-3b34e58d320f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:31.391560 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:31.451666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eac444c6-eae2-4ac6-826e-3b34e58d320f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ccc24707b2942f8abbb655154d9c8a2'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:31.452640 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eac444c6-eae2-4ac6-826e-3b34e58d320f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ccc24707b2942f8abbb655154d9c8a2'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:31.453429 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eac444c6-eae2-4ac6-826e-3b34e58d320f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ccc24707b2942f8abbb655154d9c8a2'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:31.471921 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eac444c6-eae2-4ac6-826e-3b34e58d320f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ccc24707b2942f8abbb655154d9c8a2'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:31.481866 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eac444c6-eae2-4ac6-826e-3b34e58d320f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ccc24707b2942f8abbb655154d9c8a2'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:31.515341 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1319/2660] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:31 2022] DELETE /identity/v3/users/5ccc24707b2942f8abbb655154d9c8a2 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:32.034978 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c1a0f-587d-419d-a002-dddbf0df7345 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:32.036639 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bb5c1a0f-587d-419d-a002-dddbf0df7345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:32.062439 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c1a0f-587d-419d-a002-dddbf0df7345 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:32.201211 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:32.224446 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f058a-15e5-47b0-a5e2-4d4e1a4c5b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:32.225361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f058a-15e5-47b0-a5e2-4d4e1a4c5b65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:32.225648 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f058a-15e5-47b0-a5e2-4d4e1a4c5b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:32.225923 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f058a-15e5-47b0-a5e2-4d4e1a4c5b65 admin admin] PATH_INFO: `/v3/projects/bcb78fcf0b604a068b358c61f8e6b54c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:32.228429 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f058a-15e5-47b0-a5e2-4d4e1a4c5b65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcb78fcf0b604a068b358c61f8e6b54c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:32.230399 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f058a-15e5-47b0-a5e2-4d4e1a4c5b65 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:32.331362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-033f058a-15e5-47b0-a5e2-4d4e1a4c5b65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcb78fcf0b604a068b358c61f8e6b54c', 'request_id': 'req-033f058a-15e5-47b0-a5e2-4d4e1a4c5b65'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:32.342824 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-033f058a-15e5-47b0-a5e2-4d4e1a4c5b65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcb78fcf0b604a068b358c61f8e6b54c', 'request_id': 'req-033f058a-15e5-47b0-a5e2-4d4e1a4c5b65'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:32.364356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-033f058a-15e5-47b0-a5e2-4d4e1a4c5b65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcb78fcf0b604a068b358c61f8e6b54c', 'request_id': 'req-033f058a-15e5-47b0-a5e2-4d4e1a4c5b65'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:32.380866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-033f058a-15e5-47b0-a5e2-4d4e1a4c5b65 admin admin] The token cache is being invalidate because project bcb78fcf0b604a068b358c61f8e6b54c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:32.381315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-033f058a-15e5-47b0-a5e2-4d4e1a4c5b65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:32.382078 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-033f058a-15e5-47b0-a5e2-4d4e1a4c5b65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:32.384146 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1342/2661] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:32 2022] DELETE /identity/v3/projects/bcb78fcf0b604a068b358c61f8e6b54c => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:32.388083 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac444c6-eae2-4ac6-826e-3b34e58d320f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:32.389780 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-eac444c6-eae2-4ac6-826e-3b34e58d320f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:32.414203 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac444c6-eae2-4ac6-826e-3b34e58d320f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:32.545957 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e9e3e-0ab9-4145-82aa-c7b129911073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:32.546917 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e9e3e-0ab9-4145-82aa-c7b129911073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:32.547353 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e9e3e-0ab9-4145-82aa-c7b129911073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:32.547738 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e9e3e-0ab9-4145-82aa-c7b129911073 admin admin] PATH_INFO: `/v3/users/7d92d807159a43fb94ee8967fca7facb` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:32.549128 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e9e3e-0ab9-4145-82aa-c7b129911073 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d92d807159a43fb94ee8967fca7facb)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:32.551852 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e9e3e-0ab9-4145-82aa-c7b129911073 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:32.631413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-671e9e3e-0ab9-4145-82aa-c7b129911073 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d92d807159a43fb94ee8967fca7facb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:32.632138 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-671e9e3e-0ab9-4145-82aa-c7b129911073 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d92d807159a43fb94ee8967fca7facb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:32.633195 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-671e9e3e-0ab9-4145-82aa-c7b129911073 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d92d807159a43fb94ee8967fca7facb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:32.666042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-671e9e3e-0ab9-4145-82aa-c7b129911073 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d92d807159a43fb94ee8967fca7facb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:32.681553 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-671e9e3e-0ab9-4145-82aa-c7b129911073 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d92d807159a43fb94ee8967fca7facb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:32.701310 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1320/2662] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:32 2022] DELETE /identity/v3/users/7d92d807159a43fb94ee8967fca7facb => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:33.421053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f058a-15e5-47b0-a5e2-4d4e1a4c5b65 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:33.422543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-033f058a-15e5-47b0-a5e2-4d4e1a4c5b65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:33.443364 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f058a-15e5-47b0-a5e2-4d4e1a4c5b65 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:33.550422 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86458f4-5bc3-4a10-9ee9-da75278b509d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:33.552849 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86458f4-5bc3-4a10-9ee9-da75278b509d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:33.553148 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86458f4-5bc3-4a10-9ee9-da75278b509d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:33.553426 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86458f4-5bc3-4a10-9ee9-da75278b509d admin admin] PATH_INFO: `/v3/projects/fae7d52fd1ea4abd84d3766f147b362d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:33.554912 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86458f4-5bc3-4a10-9ee9-da75278b509d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fae7d52fd1ea4abd84d3766f147b362d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:33.557116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86458f4-5bc3-4a10-9ee9-da75278b509d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:33.635862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f86458f4-5bc3-4a10-9ee9-da75278b509d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fae7d52fd1ea4abd84d3766f147b362d', 'request_id': 'req-f86458f4-5bc3-4a10-9ee9-da75278b509d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:33.646212 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f86458f4-5bc3-4a10-9ee9-da75278b509d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fae7d52fd1ea4abd84d3766f147b362d', 'request_id': 'req-f86458f4-5bc3-4a10-9ee9-da75278b509d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:33.655511 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f86458f4-5bc3-4a10-9ee9-da75278b509d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fae7d52fd1ea4abd84d3766f147b362d', 'request_id': 'req-f86458f4-5bc3-4a10-9ee9-da75278b509d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:33.670424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f86458f4-5bc3-4a10-9ee9-da75278b509d admin admin] The token cache is being invalidate because project fae7d52fd1ea4abd84d3766f147b362d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:33.670824 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f86458f4-5bc3-4a10-9ee9-da75278b509d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:33.671677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f86458f4-5bc3-4a10-9ee9-da75278b509d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:33.673386 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1343/2663] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:33 2022] DELETE /identity/v3/projects/fae7d52fd1ea4abd84d3766f147b362d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:33.677359 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599572d-5034-4d65-b2ec-cbe65f428d88 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:33.678184 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599572d-5034-4d65-b2ec-cbe65f428d88 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:33.678663 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599572d-5034-4d65-b2ec-cbe65f428d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:33.679118 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26a220-44e3-4794-8257-37e799bf92e3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:33.679462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26a220-44e3-4794-8257-37e799bf92e3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:33.679775 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26a220-44e3-4794-8257-37e799bf92e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:33.712570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:33.714226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2d26a220-44e3-4794-8257-37e799bf92e3 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:33.716029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2d26a220-44e3-4794-8257-37e799bf92e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:33.716773 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-0599572d-5034-4d65-b2ec-cbe65f428d88 None None] MFA Rules not processed for user `d7eda7675d7648c3aff7e869a5718358`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:33.719044 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0599572d-5034-4d65-b2ec-cbe65f428d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:33.729898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:33.803194 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1344/2664] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:33.806630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60104c-3ced-42d3-aeb1-762b945640d9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:33.807187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60104c-3ced-42d3-aeb1-762b945640d9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:33.807564 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60104c-3ced-42d3-aeb1-762b945640d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:33.824693 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1321/2665] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:33.836520 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3e60104c-3ced-42d3-aeb1-762b945640d9 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:33.839354 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599572d-5034-4d65-b2ec-cbe65f428d88 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:33.839821 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3e60104c-3ced-42d3-aeb1-762b945640d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:33.841154 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0599572d-5034-4d65-b2ec-cbe65f428d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:33.850109 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1345/2666] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:33.854741 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60104c-3ced-42d3-aeb1-762b945640d9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:33.865069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60104c-3ced-42d3-aeb1-762b945640d9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:33.870149 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee52e77-4bc3-4788-8753-1149f6b2b2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:33.870879 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee52e77-4bc3-4788-8753-1149f6b2b2ef admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:33.871160 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee52e77-4bc3-4788-8753-1149f6b2b2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:33.871634 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee52e77-4bc3-4788-8753-1149f6b2b2ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:33.872189 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee52e77-4bc3-4788-8753-1149f6b2b2ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:23:33.872614 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee52e77-4bc3-4788-8753-1149f6b2b2ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:33.874234 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee52e77-4bc3-4788-8753-1149f6b2b2ef admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:33.887856 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1346/2667] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:23:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:23:33.892571 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee52e77-4bc3-4788-8753-1149f6b2b2ef admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:33.903214 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee52e77-4bc3-4788-8753-1149f6b2b2ef admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:33.908146 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597e2a6-2725-4517-8a57-18eed92e7139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:33.908827 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597e2a6-2725-4517-8a57-18eed92e7139 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:33.909210 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597e2a6-2725-4517-8a57-18eed92e7139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:33.909370 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597e2a6-2725-4517-8a57-18eed92e7139 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:33.910074 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e597e2a6-2725-4517-8a57-18eed92e7139 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:33.912208 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e597e2a6-2725-4517-8a57-18eed92e7139 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:33.914741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599572d-5034-4d65-b2ec-cbe65f428d88 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:33.923659 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f015e-08fe-437e-bc6e-ea12003cf4f6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b43db1fd0b494a3f8588b498b585f247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:33.924223 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f015e-08fe-437e-bc6e-ea12003cf4f6 service glance] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:33.924561 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f015e-08fe-437e-bc6e-ea12003cf4f6 service glance] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:33.924902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f015e-08fe-437e-bc6e-ea12003cf4f6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:33.928613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5d6f015e-08fe-437e-bc6e-ea12003cf4f6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:33.945594 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1347/2668] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:33 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:33.951038 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597e2a6-2725-4517-8a57-18eed92e7139 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:34.018561 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f015e-08fe-437e-bc6e-ea12003cf4f6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:34.020131 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f015e-08fe-437e-bc6e-ea12003cf4f6 service glance] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:34.030443 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1322/2669] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:23:34.031237 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597e2a6-2725-4517-8a57-18eed92e7139 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:34.037261 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78187a0-efad-40d9-b3ac-6293bb65b91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:34.037942 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78187a0-efad-40d9-b3ac-6293bb65b91d admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:34.038224 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78187a0-efad-40d9-b3ac-6293bb65b91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:34.038500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78187a0-efad-40d9-b3ac-6293bb65b91d admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:34.039392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78187a0-efad-40d9-b3ac-6293bb65b91d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:34.041368 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78187a0-efad-40d9-b3ac-6293bb65b91d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:34.065334 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1348/2670] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:33 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:34.069442 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f015e-08fe-437e-bc6e-ea12003cf4f6 service glance] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:34.093091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f015e-08fe-437e-bc6e-ea12003cf4f6 service glance] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:34.102411 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d5bfc-62b9-46c1-acce-380d92f863e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:34.103251 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d5bfc-62b9-46c1-acce-380d92f863e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:34.103544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d5bfc-62b9-46c1-acce-380d92f863e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:34.103864 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d5bfc-62b9-46c1-acce-380d92f863e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:34.104634 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d5bfc-62b9-46c1-acce-380d92f863e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:34.106266 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d5bfc-62b9-46c1-acce-380d92f863e9 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:34.119678 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1323/2671] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:34.122598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78187a0-efad-40d9-b3ac-6293bb65b91d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:34.133686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78187a0-efad-40d9-b3ac-6293bb65b91d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:34.139889 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cf705-8978-45b3-808e-b6038943eb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:34.140636 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cf705-8978-45b3-808e-b6038943eb6c admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:34.140939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cf705-8978-45b3-808e-b6038943eb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:34.141179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cf705-8978-45b3-808e-b6038943eb6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:34.141964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4cf705-8978-45b3-808e-b6038943eb6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:34.143662 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4cf705-8978-45b3-808e-b6038943eb6c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:34.147596 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e3d07-7b5b-449e-9c9a-a48ed998d3cb None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:34.147918 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e3d07-7b5b-449e-9c9a-a48ed998d3cb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:34.148199 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e3d07-7b5b-449e-9c9a-a48ed998d3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:34.155631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1349/2672] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:23:34.159334 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cf705-8978-45b3-808e-b6038943eb6c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:34.168008 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-df6e3d07-7b5b-449e-9c9a-a48ed998d3cb None None] MFA Rules not processed for user `e802da0b078e457da466a9a7b0f15085`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:34.169231 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-df6e3d07-7b5b-449e-9c9a-a48ed998d3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:34.170415 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cf705-8978-45b3-808e-b6038943eb6c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:34.175173 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d0839-4fbe-42a6-9ee0-dfe21099dc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:34.176187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d0839-4fbe-42a6-9ee0-dfe21099dc83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:34.176633 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d0839-4fbe-42a6-9ee0-dfe21099dc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:34.176980 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d0839-4fbe-42a6-9ee0-dfe21099dc83 admin admin] PATH_INFO: `/v3/projects/7062fd34ab3b4ab0b0e179f2eb1758da/users/09b84d25163649a88a288763e80d299c/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:34.194270 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d0839-4fbe-42a6-9ee0-dfe21099dc83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7062fd34ab3b4ab0b0e179f2eb1758da, user_id=09b84d25163649a88a288763e80d299c, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:34.197010 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d0839-4fbe-42a6-9ee0-dfe21099dc83 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:34.212904 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:34.214196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1350/2673] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:34 2022] PUT /identity/v3/projects/7062fd34ab3b4ab0b0e179f2eb1758da/users/09b84d25163649a88a288763e80d299c/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:34.225773 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150861fe-57dc-479d-b646-b9e39f50f465 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:34.226055 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150861fe-57dc-479d-b646-b9e39f50f465 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:34.226329 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150861fe-57dc-479d-b646-b9e39f50f465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:34.238144 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1324/2674] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Jun 27 21:23:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:34.254798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-150861fe-57dc-479d-b646-b9e39f50f465 None None] MFA Rules not processed for user `09b84d25163649a88a288763e80d299c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:34.256357 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-150861fe-57dc-479d-b646-b9e39f50f465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:34.261856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e3d07-7b5b-449e-9c9a-a48ed998d3cb None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:34.264392 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-df6e3d07-7b5b-449e-9c9a-a48ed998d3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:34.272480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:34.326963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1351/2675] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:34.331785 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150861fe-57dc-479d-b646-b9e39f50f465 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:34.354208 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e3d07-7b5b-449e-9c9a-a48ed998d3cb None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:34.401433 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150861fe-57dc-479d-b646-b9e39f50f465 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:34.406090 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898f10d-4449-4f96-b186-a7d290df9268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:34.406791 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898f10d-4449-4f96-b186-a7d290df9268 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:34.407075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898f10d-4449-4f96-b186-a7d290df9268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:34.407351 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898f10d-4449-4f96-b186-a7d290df9268 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:34.408085 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898f10d-4449-4f96-b186-a7d290df9268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:34.410005 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898f10d-4449-4f96-b186-a7d290df9268 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:34.417370 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc03a0-519c-4e3a-9b0b-25cc4d6cfc9b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28acb0322114aebafcf62bdf9a346df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:34.418023 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc03a0-519c-4e3a-9b0b-25cc4d6cfc9b service swift] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:34.418335 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc03a0-519c-4e3a-9b0b-25cc4d6cfc9b service swift] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:34.418610 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc03a0-519c-4e3a-9b0b-25cc4d6cfc9b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:34.436060 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1352/2676] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:34 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:34.440042 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898f10d-4449-4f96-b186-a7d290df9268 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:34.480070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc03a0-519c-4e3a-9b0b-25cc4d6cfc9b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:34.481645 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc03a0-519c-4e3a-9b0b-25cc4d6cfc9b service swift] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:34.490523 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1325/2677] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Jun 27 21:23:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 21:23:34.504252 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898f10d-4449-4f96-b186-a7d290df9268 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:34.509477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fca02e-bb2e-4637-96cc-4d0e6e8f9a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:34.510781 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fca02e-bb2e-4637-96cc-4d0e6e8f9a02 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:34.510781 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fca02e-bb2e-4637-96cc-4d0e6e8f9a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:34.511080 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fca02e-bb2e-4637-96cc-4d0e6e8f9a02 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:34.511903 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fca02e-bb2e-4637-96cc-4d0e6e8f9a02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:34.514180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fca02e-bb2e-4637-96cc-4d0e6e8f9a02 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:34.542810 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1353/2678] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:34 2022] POST /identity/v3/users => generated 298 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:34.546481 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc03a0-519c-4e3a-9b0b-25cc4d6cfc9b service swift] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:34.567546 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc03a0-519c-4e3a-9b0b-25cc4d6cfc9b service swift] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:34.577079 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54bf39-dec8-42fd-92cb-c00cc9beffc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:34.577858 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54bf39-dec8-42fd-92cb-c00cc9beffc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:34.578150 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54bf39-dec8-42fd-92cb-c00cc9beffc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:34.578432 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54bf39-dec8-42fd-92cb-c00cc9beffc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:34.579209 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54bf39-dec8-42fd-92cb-c00cc9beffc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:34.581036 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54bf39-dec8-42fd-92cb-c00cc9beffc8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:34.602415 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1326/2679] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:34.603920 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fca02e-bb2e-4637-96cc-4d0e6e8f9a02 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:34.616155 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fca02e-bb2e-4637-96cc-4d0e6e8f9a02 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:34.620643 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842e234-6590-4f9c-b125-7955dfb47a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:34.621624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842e234-6590-4f9c-b125-7955dfb47a0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:34.621906 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842e234-6590-4f9c-b125-7955dfb47a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:34.622180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842e234-6590-4f9c-b125-7955dfb47a0e admin admin] PATH_INFO: `/v3/projects/87ec71aec2db4d89924ccbd1f1d7459d/users/61ad04ccf864491ca9c23a4e0f3c2c68/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:34.646249 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842e234-6590-4f9c-b125-7955dfb47a0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=87ec71aec2db4d89924ccbd1f1d7459d, user_id=61ad04ccf864491ca9c23a4e0f3c2c68, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:34.649209 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842e234-6590-4f9c-b125-7955dfb47a0e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:34.677722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1354/2680] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:34 2022] PUT /identity/v3/projects/87ec71aec2db4d89924ccbd1f1d7459d/users/61ad04ccf864491ca9c23a4e0f3c2c68/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:34.679594 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54bf39-dec8-42fd-92cb-c00cc9beffc8 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:34.690492 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54bf39-dec8-42fd-92cb-c00cc9beffc8 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:34.714971 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:34.769595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a4362-89f4-473b-b4b7-34b369f06bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:34.770415 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a4362-89f4-473b-b4b7-34b369f06bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:34.770700 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a4362-89f4-473b-b4b7-34b369f06bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:34.770979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a4362-89f4-473b-b4b7-34b369f06bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:34.771794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a4362-89f4-473b-b4b7-34b369f06bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:34.775719 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a4362-89f4-473b-b4b7-34b369f06bf0 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:34.787005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1327/2681] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:34.792108 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842e234-6590-4f9c-b125-7955dfb47a0e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:34.803024 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842e234-6590-4f9c-b125-7955dfb47a0e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:34.807760 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb569f14-7b31-49ea-9235-217a86999d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:34.808679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb569f14-7b31-49ea-9235-217a86999d46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:34.808950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb569f14-7b31-49ea-9235-217a86999d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:34.809220 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb569f14-7b31-49ea-9235-217a86999d46 admin admin] PATH_INFO: `/v3/projects/87ec71aec2db4d89924ccbd1f1d7459d/users/61ad04ccf864491ca9c23a4e0f3c2c68/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:34.811161 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb569f14-7b31-49ea-9235-217a86999d46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87ec71aec2db4d89924ccbd1f1d7459d, user_id=61ad04ccf864491ca9c23a4e0f3c2c68, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:34.813862 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb569f14-7b31-49ea-9235-217a86999d46 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:34.817401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:34.827727 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1355/2682] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:34 2022] PUT /identity/v3/projects/87ec71aec2db4d89924ccbd1f1d7459d/users/61ad04ccf864491ca9c23a4e0f3c2c68/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:34.909605 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e88ece-6993-4e47-89c8-88552eaf9df3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:34.910481 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e88ece-6993-4e47-89c8-88552eaf9df3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:34.910913 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e88ece-6993-4e47-89c8-88552eaf9df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:34.953537 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-44e88ece-6993-4e47-89c8-88552eaf9df3 None None] MFA Rules not processed for user `61ad04ccf864491ca9c23a4e0f3c2c68`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:34.955192 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-44e88ece-6993-4e47-89c8-88552eaf9df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:35.047664 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1328/2683] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:35.062719 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb569f14-7b31-49ea-9235-217a86999d46 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:35.064552 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bb569f14-7b31-49ea-9235-217a86999d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:35.137476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb569f14-7b31-49ea-9235-217a86999d46 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:35.141782 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8ed2f-289e-4ba7-b916-8e9661c19198 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:35.142556 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8ed2f-289e-4ba7-b916-8e9661c19198 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:35.142948 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8ed2f-289e-4ba7-b916-8e9661c19198 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:35.143319 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8ed2f-289e-4ba7-b916-8e9661c19198 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:35.156114 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8ed2f-289e-4ba7-b916-8e9661c19198 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:35.158577 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8ed2f-289e-4ba7-b916-8e9661c19198 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:35.172284 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1356/2684] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:35.276931 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e88ece-6993-4e47-89c8-88552eaf9df3 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:35.293851 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e88ece-6993-4e47-89c8-88552eaf9df3 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:35.301035 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7de91-c010-4805-be3f-e7b2d7ffb7b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:35.306246 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7de91-c010-4805-be3f-e7b2d7ffb7b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:35.307134 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7de91-c010-4805-be3f-e7b2d7ffb7b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:35.307431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7de91-c010-4805-be3f-e7b2d7ffb7b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:35.388137 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7de91-c010-4805-be3f-e7b2d7ffb7b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:35.389702 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7de91-c010-4805-be3f-e7b2d7ffb7b1 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:35.409047 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1329/2685] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:35.774968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8ed2f-289e-4ba7-b916-8e9661c19198 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:35.860602 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8ed2f-289e-4ba7-b916-8e9661c19198 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:35.866724 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5812e7c-57e0-4bd5-8591-6844ef5f048b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:35.867363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5812e7c-57e0-4bd5-8591-6844ef5f048b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:35.867677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5812e7c-57e0-4bd5-8591-6844ef5f048b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:35.868055 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5812e7c-57e0-4bd5-8591-6844ef5f048b admin admin] PATH_INFO: `/v3/users/09b84d25163649a88a288763e80d299c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:35.869514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5812e7c-57e0-4bd5-8591-6844ef5f048b admin admin] RBAC: Authorizing `identity:delete_user(user_id=09b84d25163649a88a288763e80d299c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:35.872216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5812e7c-57e0-4bd5-8591-6844ef5f048b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:35.936981 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c5812e7c-57e0-4bd5-8591-6844ef5f048b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09b84d25163649a88a288763e80d299c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:35.937660 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c5812e7c-57e0-4bd5-8591-6844ef5f048b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09b84d25163649a88a288763e80d299c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:35.938325 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c5812e7c-57e0-4bd5-8591-6844ef5f048b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09b84d25163649a88a288763e80d299c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:35.954509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c5812e7c-57e0-4bd5-8591-6844ef5f048b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09b84d25163649a88a288763e80d299c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:35.964384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c5812e7c-57e0-4bd5-8591-6844ef5f048b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09b84d25163649a88a288763e80d299c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:35.996083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1357/2686] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:35 2022] DELETE /identity/v3/users/09b84d25163649a88a288763e80d299c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:36.005796 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f2952-074d-4624-b595-123db4b091a9 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:36.007590 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f2952-074d-4624-b595-123db4b091a9 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:36.007938 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f2952-074d-4624-b595-123db4b091a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:36.047671 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-8e8f2952-074d-4624-b595-123db4b091a9 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:36.054069 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8e8f2952-074d-4624-b595-123db4b091a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:36.103484 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:36.143444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1330/2687] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:36.154563 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5812e7c-57e0-4bd5-8591-6844ef5f048b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:36.156694 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c5812e7c-57e0-4bd5-8591-6844ef5f048b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:36.184271 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:36.256569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5812e7c-57e0-4bd5-8591-6844ef5f048b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:36.266091 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793b31e-84bd-4d4d-80e9-243b38cd979d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:36.266949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793b31e-84bd-4d4d-80e9-243b38cd979d service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:36.267494 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793b31e-84bd-4d4d-80e9-243b38cd979d service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:36.267805 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793b31e-84bd-4d4d-80e9-243b38cd979d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:36.278527 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793b31e-84bd-4d4d-80e9-243b38cd979d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:36.280316 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793b31e-84bd-4d4d-80e9-243b38cd979d service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:36.292700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1358/2688] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:36.661508 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f2952-074d-4624-b595-123db4b091a9 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:36.662969 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8e8f2952-074d-4624-b595-123db4b091a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:36.683967 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f2952-074d-4624-b595-123db4b091a9 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:36.691404 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745d714-7bda-41ef-9c1e-ce212fb33152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:36.692504 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745d714-7bda-41ef-9c1e-ce212fb33152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:36.692915 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745d714-7bda-41ef-9c1e-ce212fb33152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:36.693295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745d714-7bda-41ef-9c1e-ce212fb33152 admin admin] PATH_INFO: `/v3/projects/7062fd34ab3b4ab0b0e179f2eb1758da` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:36.694799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745d714-7bda-41ef-9c1e-ce212fb33152 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7062fd34ab3b4ab0b0e179f2eb1758da)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:36.697110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745d714-7bda-41ef-9c1e-ce212fb33152 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:36.780485 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6745d714-7bda-41ef-9c1e-ce212fb33152 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7062fd34ab3b4ab0b0e179f2eb1758da', 'request_id': 'req-6745d714-7bda-41ef-9c1e-ce212fb33152'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:36.795990 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6745d714-7bda-41ef-9c1e-ce212fb33152 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7062fd34ab3b4ab0b0e179f2eb1758da', 'request_id': 'req-6745d714-7bda-41ef-9c1e-ce212fb33152'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:36.812048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6745d714-7bda-41ef-9c1e-ce212fb33152 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7062fd34ab3b4ab0b0e179f2eb1758da', 'request_id': 'req-6745d714-7bda-41ef-9c1e-ce212fb33152'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:36.839760 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6745d714-7bda-41ef-9c1e-ce212fb33152 admin admin] The token cache is being invalidate because project 7062fd34ab3b4ab0b0e179f2eb1758da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:36.841727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6745d714-7bda-41ef-9c1e-ce212fb33152 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:36.844619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6745d714-7bda-41ef-9c1e-ce212fb33152 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:36.851230 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1331/2689] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:36 2022] DELETE /identity/v3/projects/7062fd34ab3b4ab0b0e179f2eb1758da => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:36.855466 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793b31e-84bd-4d4d-80e9-243b38cd979d service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:36.871596 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793b31e-84bd-4d4d-80e9-243b38cd979d service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:37.019863 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ffe1a-1dbc-4cd6-94cb-96ef4ec9e273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:37.021295 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ffe1a-1dbc-4cd6-94cb-96ef4ec9e273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:37.021963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ffe1a-1dbc-4cd6-94cb-96ef4ec9e273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:37.022426 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ffe1a-1dbc-4cd6-94cb-96ef4ec9e273 admin admin] PATH_INFO: `/v3/users/61ad04ccf864491ca9c23a4e0f3c2c68` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:37.024108 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ffe1a-1dbc-4cd6-94cb-96ef4ec9e273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61ad04ccf864491ca9c23a4e0f3c2c68)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:37.026847 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ffe1a-1dbc-4cd6-94cb-96ef4ec9e273 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:37.154024 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7d2ffe1a-1dbc-4cd6-94cb-96ef4ec9e273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61ad04ccf864491ca9c23a4e0f3c2c68'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:37.154767 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7d2ffe1a-1dbc-4cd6-94cb-96ef4ec9e273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61ad04ccf864491ca9c23a4e0f3c2c68'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:37.155835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7d2ffe1a-1dbc-4cd6-94cb-96ef4ec9e273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61ad04ccf864491ca9c23a4e0f3c2c68'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:37.175357 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7d2ffe1a-1dbc-4cd6-94cb-96ef4ec9e273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61ad04ccf864491ca9c23a4e0f3c2c68'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:37.194651 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7d2ffe1a-1dbc-4cd6-94cb-96ef4ec9e273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61ad04ccf864491ca9c23a4e0f3c2c68'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:37.224775 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1359/2690] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:36 2022] DELETE /identity/v3/users/61ad04ccf864491ca9c23a4e0f3c2c68 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:37.705928 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745d714-7bda-41ef-9c1e-ce212fb33152 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:37.709053 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6745d714-7bda-41ef-9c1e-ce212fb33152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:37.739265 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745d714-7bda-41ef-9c1e-ce212fb33152 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:37.827501 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6ffd2-e02a-4e24-8207-2c7c1b8f6a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:37.828896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6ffd2-e02a-4e24-8207-2c7c1b8f6a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:37.829328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6ffd2-e02a-4e24-8207-2c7c1b8f6a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:37.830135 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6ffd2-e02a-4e24-8207-2c7c1b8f6a78 admin admin] PATH_INFO: `/v3/projects/87ec71aec2db4d89924ccbd1f1d7459d` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:37.831348 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6ffd2-e02a-4e24-8207-2c7c1b8f6a78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87ec71aec2db4d89924ccbd1f1d7459d)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:37.833961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6ffd2-e02a-4e24-8207-2c7c1b8f6a78 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:37.896181 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-18a6ffd2-e02a-4e24-8207-2c7c1b8f6a78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87ec71aec2db4d89924ccbd1f1d7459d', 'request_id': 'req-18a6ffd2-e02a-4e24-8207-2c7c1b8f6a78'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:37.906998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-18a6ffd2-e02a-4e24-8207-2c7c1b8f6a78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87ec71aec2db4d89924ccbd1f1d7459d', 'request_id': 'req-18a6ffd2-e02a-4e24-8207-2c7c1b8f6a78'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:37.913870 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-18a6ffd2-e02a-4e24-8207-2c7c1b8f6a78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87ec71aec2db4d89924ccbd1f1d7459d', 'request_id': 'req-18a6ffd2-e02a-4e24-8207-2c7c1b8f6a78'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:37.915110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:37.929425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-18a6ffd2-e02a-4e24-8207-2c7c1b8f6a78 admin admin] The token cache is being invalidate because project 87ec71aec2db4d89924ccbd1f1d7459d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:37.930377 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-18a6ffd2-e02a-4e24-8207-2c7c1b8f6a78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:37.931289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-18a6ffd2-e02a-4e24-8207-2c7c1b8f6a78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:37.933191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1332/2691] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:37 2022] DELETE /identity/v3/projects/87ec71aec2db4d89924ccbd1f1d7459d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:37.938610 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029900c3-9f5f-4e1d-8511-59426cc25c54 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:37.939140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029900c3-9f5f-4e1d-8511-59426cc25c54 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:37.939140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029900c3-9f5f-4e1d-8511-59426cc25c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:37.957586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-029900c3-9f5f-4e1d-8511-59426cc25c54 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:37.958839 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-029900c3-9f5f-4e1d-8511-59426cc25c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:37.998283 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:38.032260 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1360/2692] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:38.040454 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7acaf-7ed9-447b-8c99-18378eff6f14 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:38.040738 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7acaf-7ed9-447b-8c99-18378eff6f14 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:38.041070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7acaf-7ed9-447b-8c99-18378eff6f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:38.061537 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ffb7acaf-7ed9-447b-8c99-18378eff6f14 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:38.062701 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ffb7acaf-7ed9-447b-8c99-18378eff6f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:38.071062 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1333/2693] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:38.075886 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029900c3-9f5f-4e1d-8511-59426cc25c54 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:38.086369 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029900c3-9f5f-4e1d-8511-59426cc25c54 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:38.091832 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30b849-ba4a-4308-bec9-71e05227fc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:38.092536 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30b849-ba4a-4308-bec9-71e05227fc8b admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:38.092989 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30b849-ba4a-4308-bec9-71e05227fc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:38.093267 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30b849-ba4a-4308-bec9-71e05227fc8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:38.094501 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30b849-ba4a-4308-bec9-71e05227fc8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:23:38.094501 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30b849-ba4a-4308-bec9-71e05227fc8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:38.095996 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30b849-ba4a-4308-bec9-71e05227fc8b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:38.109585 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1361/2694] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:23:38.114788 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7acaf-7ed9-447b-8c99-18378eff6f14 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:38.123793 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7acaf-7ed9-447b-8c99-18378eff6f14 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:38.128726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14464847-31a8-405b-8623-66b4a733e17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:38.129395 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14464847-31a8-405b-8623-66b4a733e17f admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:38.129675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14464847-31a8-405b-8623-66b4a733e17f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:38.129953 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14464847-31a8-405b-8623-66b4a733e17f admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:38.130640 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14464847-31a8-405b-8623-66b4a733e17f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:38.132577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14464847-31a8-405b-8623-66b4a733e17f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:38.171373 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1334/2695] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:38 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:38.175707 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30b849-ba4a-4308-bec9-71e05227fc8b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:38.188201 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30b849-ba4a-4308-bec9-71e05227fc8b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:38.250146 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08719840-dce1-44da-9992-12e2464cd796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:38.250831 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08719840-dce1-44da-9992-12e2464cd796 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:38.251114 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08719840-dce1-44da-9992-12e2464cd796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:38.251873 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08719840-dce1-44da-9992-12e2464cd796 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:38.252148 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08719840-dce1-44da-9992-12e2464cd796 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:38.254461 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08719840-dce1-44da-9992-12e2464cd796 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:38.280468 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1362/2696] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:38 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:38.286534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14464847-31a8-405b-8623-66b4a733e17f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:38.297846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14464847-31a8-405b-8623-66b4a733e17f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:38.309212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ea1fe-6bbd-4ba0-8f42-e6d3c183efad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:38.310250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ea1fe-6bbd-4ba0-8f42-e6d3c183efad admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:38.310793 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ea1fe-6bbd-4ba0-8f42-e6d3c183efad admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:38.311221 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ea1fe-6bbd-4ba0-8f42-e6d3c183efad admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:38.312205 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ea1fe-6bbd-4ba0-8f42-e6d3c183efad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:38.314352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ea1fe-6bbd-4ba0-8f42-e6d3c183efad admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:38.327402 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1335/2697] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:38.331322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08719840-dce1-44da-9992-12e2464cd796 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:38.345009 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08719840-dce1-44da-9992-12e2464cd796 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:38.350655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd7658-ef08-40f4-a5e4-2d87415a6f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:38.354331 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd7658-ef08-40f4-a5e4-2d87415a6f71 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:38.354504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd7658-ef08-40f4-a5e4-2d87415a6f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:38.354724 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd7658-ef08-40f4-a5e4-2d87415a6f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:38.355468 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd7658-ef08-40f4-a5e4-2d87415a6f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:38.358584 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd7658-ef08-40f4-a5e4-2d87415a6f71 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:38.373524 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1363/2698] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:23:38.378001 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ea1fe-6bbd-4ba0-8f42-e6d3c183efad admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:38.388361 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ea1fe-6bbd-4ba0-8f42-e6d3c183efad admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:38.393029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99737aa-0176-41ba-ae32-28c64ffbe843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:38.393938 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99737aa-0176-41ba-ae32-28c64ffbe843 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:38.394212 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99737aa-0176-41ba-ae32-28c64ffbe843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:38.394492 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99737aa-0176-41ba-ae32-28c64ffbe843 admin admin] PATH_INFO: `/v3/projects/cc09b2f148714e17a620de8e6bee326e/users/ea6802744a85492cbe183b7c7e237411/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:38.412954 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99737aa-0176-41ba-ae32-28c64ffbe843 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc09b2f148714e17a620de8e6bee326e, user_id=ea6802744a85492cbe183b7c7e237411, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:38.415769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99737aa-0176-41ba-ae32-28c64ffbe843 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:38.436564 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1336/2699] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:38 2022] PUT /identity/v3/projects/cc09b2f148714e17a620de8e6bee326e/users/ea6802744a85492cbe183b7c7e237411/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:38.448456 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad100a-229e-4c5e-a436-2100a7585e67 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:38.449163 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad100a-229e-4c5e-a436-2100a7585e67 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:38.449891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad100a-229e-4c5e-a436-2100a7585e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:38.486335 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a3ad100a-229e-4c5e-a436-2100a7585e67 None None] MFA Rules not processed for user `ea6802744a85492cbe183b7c7e237411`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:38.487722 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a3ad100a-229e-4c5e-a436-2100a7585e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:38.572977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1364/2700] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:38.583855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99737aa-0176-41ba-ae32-28c64ffbe843 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:38.585609 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e99737aa-0176-41ba-ae32-28c64ffbe843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:38.671748 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99737aa-0176-41ba-ae32-28c64ffbe843 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:38.675972 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d591f9cf-709c-4bb2-97c9-f88117259df0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:38.676864 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591f9cf-709c-4bb2-97c9-f88117259df0 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:38.677213 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591f9cf-709c-4bb2-97c9-f88117259df0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:38.677555 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591f9cf-709c-4bb2-97c9-f88117259df0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:38.688241 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d591f9cf-709c-4bb2-97c9-f88117259df0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:38.689810 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d591f9cf-709c-4bb2-97c9-f88117259df0 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:38.705855 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1337/2701] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:44.468979 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3346f-b0d2-44a3-9af5-c20f4574ff93 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:44.469555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3346f-b0d2-44a3-9af5-c20f4574ff93 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:44.469555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3346f-b0d2-44a3-9af5-c20f4574ff93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:44.488275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-67d3346f-b0d2-44a3-9af5-c20f4574ff93 None None] MFA Rules not processed for user `19e54ac48c1943b0999baeaad67a0e11`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:44.489502 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-67d3346f-b0d2-44a3-9af5-c20f4574ff93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:44.520898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:44.554936 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1365/2702] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:44.822144 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d591f9cf-709c-4bb2-97c9-f88117259df0 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:44.823714 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d591f9cf-709c-4bb2-97c9-f88117259df0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:44.888036 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d591f9cf-709c-4bb2-97c9-f88117259df0 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:44.892190 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c79a1b-1be2-42dd-9715-dc3361427172 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:44.892932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c79a1b-1be2-42dd-9715-dc3361427172 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:44.893348 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c79a1b-1be2-42dd-9715-dc3361427172 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:44.893661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c79a1b-1be2-42dd-9715-dc3361427172 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:44.895923 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-15c79a1b-1be2-42dd-9715-dc3361427172 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:44.909559 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c79a1b-1be2-42dd-9715-dc3361427172 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:44.911013 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c79a1b-1be2-42dd-9715-dc3361427172 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:44.920234 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1338/2703] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:46.667447 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54deba-ffb2-41d6-94fb-d1168d21ad93 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:46.667961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54deba-ffb2-41d6-94fb-d1168d21ad93 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:46.668180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54deba-ffb2-41d6-94fb-d1168d21ad93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:46.686927 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9c54deba-ffb2-41d6-94fb-d1168d21ad93 None None] MFA Rules not processed for user `e802da0b078e457da466a9a7b0f15085`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:46.688301 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9c54deba-ffb2-41d6-94fb-d1168d21ad93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:46.756001 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1366/2704] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Jun 27 21:23:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:46.771971 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c79a1b-1be2-42dd-9715-dc3361427172 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:46.773659 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-15c79a1b-1be2-42dd-9715-dc3361427172 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:46.837991 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:46.884977 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c79a1b-1be2-42dd-9715-dc3361427172 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:46.890472 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d935784-d436-40dd-bbfd-77fd361af84f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28acb0322114aebafcf62bdf9a346df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:46.890895 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54deba-ffb2-41d6-94fb-d1168d21ad93 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:46.891334 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d935784-d436-40dd-bbfd-77fd361af84f service swift] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:46.891707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d935784-d436-40dd-bbfd-77fd361af84f service swift] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:46.892202 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d935784-d436-40dd-bbfd-77fd361af84f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:46.892945 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9c54deba-ffb2-41d6-94fb-d1168d21ad93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:46.905856 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d935784-d436-40dd-bbfd-77fd361af84f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:46.906901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54deba-ffb2-41d6-94fb-d1168d21ad93 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:46.907498 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d935784-d436-40dd-bbfd-77fd361af84f service swift] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:46.915594 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b5d3b-ec80-44e4-a222-f397611e007e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:46.916380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b5d3b-ec80-44e4-a222-f397611e007e service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:46.916754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b5d3b-ec80-44e4-a222-f397611e007e service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:46.917053 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b5d3b-ec80-44e4-a222-f397611e007e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:46.919021 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-481b5d3b-ec80-44e4-a222-f397611e007e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:46.919714 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1339/2705] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Jun 27 21:23:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:23:47.028293 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b5d3b-ec80-44e4-a222-f397611e007e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:47.030498 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b5d3b-ec80-44e4-a222-f397611e007e service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:47.045186 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1367/2706] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:47.562430 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d935784-d436-40dd-bbfd-77fd361af84f service swift] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:47.575876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d935784-d436-40dd-bbfd-77fd361af84f service swift] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:47.588148 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f2512-fb13-4e7d-8579-414cc980e8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:47.589347 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f2512-fb13-4e7d-8579-414cc980e8ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:47.589812 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f2512-fb13-4e7d-8579-414cc980e8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:47.590330 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f2512-fb13-4e7d-8579-414cc980e8ab admin admin] PATH_INFO: `/v3/users/ea6802744a85492cbe183b7c7e237411` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:47.591919 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f2512-fb13-4e7d-8579-414cc980e8ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea6802744a85492cbe183b7c7e237411)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:47.594986 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f2512-fb13-4e7d-8579-414cc980e8ab admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:47.692675 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-139f2512-fb13-4e7d-8579-414cc980e8ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea6802744a85492cbe183b7c7e237411'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:47.693692 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-139f2512-fb13-4e7d-8579-414cc980e8ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea6802744a85492cbe183b7c7e237411'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:47.694337 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-139f2512-fb13-4e7d-8579-414cc980e8ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea6802744a85492cbe183b7c7e237411'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:47.732286 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-139f2512-fb13-4e7d-8579-414cc980e8ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea6802744a85492cbe183b7c7e237411'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:47.747748 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-139f2512-fb13-4e7d-8579-414cc980e8ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea6802744a85492cbe183b7c7e237411'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:47.771185 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1340/2707] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:47 2022] DELETE /identity/v3/users/ea6802744a85492cbe183b7c7e237411 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:47.775887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade90cd-e998-4b1a-be39-91a5b01952ae None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:47.777153 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade90cd-e998-4b1a-be39-91a5b01952ae None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:47.777705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade90cd-e998-4b1a-be39-91a5b01952ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:47.810388 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4ade90cd-e998-4b1a-be39-91a5b01952ae None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:47.812047 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4ade90cd-e998-4b1a-be39-91a5b01952ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:47.905368 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1368/2708] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:47.912708 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f2512-fb13-4e7d-8579-414cc980e8ab admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:47.914194 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-139f2512-fb13-4e7d-8579-414cc980e8ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:47.996192 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f2512-fb13-4e7d-8579-414cc980e8ab admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:48.000108 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066bc99a-3f58-4fbf-822e-a9cda08969e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:48.000793 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066bc99a-3f58-4fbf-822e-a9cda08969e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:48.001070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066bc99a-3f58-4fbf-822e-a9cda08969e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:48.001349 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066bc99a-3f58-4fbf-822e-a9cda08969e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:48.013145 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066bc99a-3f58-4fbf-822e-a9cda08969e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:48.014794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066bc99a-3f58-4fbf-822e-a9cda08969e8 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:48.025863 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1341/2709] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:48.372023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade90cd-e998-4b1a-be39-91a5b01952ae None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:48.374246 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4ade90cd-e998-4b1a-be39-91a5b01952ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:48.389986 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade90cd-e998-4b1a-be39-91a5b01952ae None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:48.396295 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8d8d2-2eca-41c6-a6ae-ca9c80b26e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:48.398758 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8d8d2-2eca-41c6-a6ae-ca9c80b26e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:48.399189 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8d8d2-2eca-41c6-a6ae-ca9c80b26e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:48.399515 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8d8d2-2eca-41c6-a6ae-ca9c80b26e7b admin admin] PATH_INFO: `/v3/projects/cc09b2f148714e17a620de8e6bee326e` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:48.401020 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8d8d2-2eca-41c6-a6ae-ca9c80b26e7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc09b2f148714e17a620de8e6bee326e)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:48.403852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8d8d2-2eca-41c6-a6ae-ca9c80b26e7b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:48.419575 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:48.504723 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-86c8d8d2-2eca-41c6-a6ae-ca9c80b26e7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc09b2f148714e17a620de8e6bee326e', 'request_id': 'req-86c8d8d2-2eca-41c6-a6ae-ca9c80b26e7b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:48.518378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-86c8d8d2-2eca-41c6-a6ae-ca9c80b26e7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc09b2f148714e17a620de8e6bee326e', 'request_id': 'req-86c8d8d2-2eca-41c6-a6ae-ca9c80b26e7b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:48.526065 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-86c8d8d2-2eca-41c6-a6ae-ca9c80b26e7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc09b2f148714e17a620de8e6bee326e', 'request_id': 'req-86c8d8d2-2eca-41c6-a6ae-ca9c80b26e7b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:48.543361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-86c8d8d2-2eca-41c6-a6ae-ca9c80b26e7b admin admin] The token cache is being invalidate because project cc09b2f148714e17a620de8e6bee326e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:48.543778 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-86c8d8d2-2eca-41c6-a6ae-ca9c80b26e7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:48.545113 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-86c8d8d2-2eca-41c6-a6ae-ca9c80b26e7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:48.552456 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1369/2710] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:48 2022] DELETE /identity/v3/projects/cc09b2f148714e17a620de8e6bee326e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:48.553419 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc94af6-0409-4ce1-a8b6-c715a58d6a0b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:48.553655 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc94af6-0409-4ce1-a8b6-c715a58d6a0b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:48.554000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc94af6-0409-4ce1-a8b6-c715a58d6a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:48.581464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-0cc94af6-0409-4ce1-a8b6-c715a58d6a0b None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:48.582736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0cc94af6-0409-4ce1-a8b6-c715a58d6a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:48.644546 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:48.714138 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba033933-5643-4f39-9a50-d6e27fbc9310 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:48.714666 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba033933-5643-4f39-9a50-d6e27fbc9310 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:48.715087 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba033933-5643-4f39-9a50-d6e27fbc9310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:48.724671 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1342/2711] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:48.735062 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ba033933-5643-4f39-9a50-d6e27fbc9310 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:48.736973 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ba033933-5643-4f39-9a50-d6e27fbc9310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:48.754897 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1370/2712] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:48.759124 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc94af6-0409-4ce1-a8b6-c715a58d6a0b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:48.769752 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc94af6-0409-4ce1-a8b6-c715a58d6a0b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:48.774740 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4db1ae-14cd-4f7b-b01d-44fb13dc6bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:48.775531 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4db1ae-14cd-4f7b-b01d-44fb13dc6bbb admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:48.775914 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4db1ae-14cd-4f7b-b01d-44fb13dc6bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:48.776310 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4db1ae-14cd-4f7b-b01d-44fb13dc6bbb admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:48.777162 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4db1ae-14cd-4f7b-b01d-44fb13dc6bbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:23:48.777544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4db1ae-14cd-4f7b-b01d-44fb13dc6bbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:48.779288 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4db1ae-14cd-4f7b-b01d-44fb13dc6bbb admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:48.792201 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1343/2713] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:23:48.804534 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba033933-5643-4f39-9a50-d6e27fbc9310 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:48.826168 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba033933-5643-4f39-9a50-d6e27fbc9310 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:48.831348 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2caab7-83f3-44d2-a9a0-e70eadd97f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:48.832062 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2caab7-83f3-44d2-a9a0-e70eadd97f24 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:48.832341 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2caab7-83f3-44d2-a9a0-e70eadd97f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:48.832617 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2caab7-83f3-44d2-a9a0-e70eadd97f24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:48.833417 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2caab7-83f3-44d2-a9a0-e70eadd97f24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:48.836667 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2caab7-83f3-44d2-a9a0-e70eadd97f24 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:48.941047 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1371/2714] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:48 2022] POST /identity/v3/projects => generated 377 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:48.942250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4db1ae-14cd-4f7b-b01d-44fb13dc6bbb admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:48.959569 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4db1ae-14cd-4f7b-b01d-44fb13dc6bbb admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:49.101925 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9d866-6ede-428d-9f08-ce14d67e518d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:49.102754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9d866-6ede-428d-9f08-ce14d67e518d admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:49.102973 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9d866-6ede-428d-9f08-ce14d67e518d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:49.103301 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9d866-6ede-428d-9f08-ce14d67e518d admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:49.103982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9d866-6ede-428d-9f08-ce14d67e518d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:49.109595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9d866-6ede-428d-9f08-ce14d67e518d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:49.167649 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1344/2715] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:48 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:49.169263 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2caab7-83f3-44d2-a9a0-e70eadd97f24 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:49.183592 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2caab7-83f3-44d2-a9a0-e70eadd97f24 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:49.190448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812f1c2-c161-4c5c-b35f-996b4060d51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:49.191769 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812f1c2-c161-4c5c-b35f-996b4060d51c admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:49.192340 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812f1c2-c161-4c5c-b35f-996b4060d51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:49.192795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812f1c2-c161-4c5c-b35f-996b4060d51c admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:49.193724 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d812f1c2-c161-4c5c-b35f-996b4060d51c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:49.195825 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d812f1c2-c161-4c5c-b35f-996b4060d51c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:49.214935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1372/2716] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:49.216006 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9d866-6ede-428d-9f08-ce14d67e518d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:49.231470 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9d866-6ede-428d-9f08-ce14d67e518d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:49.241916 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea2be4-81de-4284-b09b-0998ee65bfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:49.246587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea2be4-81de-4284-b09b-0998ee65bfb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:49.247881 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea2be4-81de-4284-b09b-0998ee65bfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:49.248455 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea2be4-81de-4284-b09b-0998ee65bfb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:49.251091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cea2be4-81de-4284-b09b-0998ee65bfb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:49.256902 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cea2be4-81de-4284-b09b-0998ee65bfb8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:49.280035 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1345/2717] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:49.284187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812f1c2-c161-4c5c-b35f-996b4060d51c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:49.305153 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812f1c2-c161-4c5c-b35f-996b4060d51c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:49.310930 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c6257-2811-4c4e-92dd-71a6a8b13f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:49.312104 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c6257-2811-4c4e-92dd-71a6a8b13f4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:49.312499 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c6257-2811-4c4e-92dd-71a6a8b13f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:49.312869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c6257-2811-4c4e-92dd-71a6a8b13f4b admin admin] PATH_INFO: `/v3/projects/84258731610646279517fb4b69e87127/users/f364ea1c990a41e8a6aa6e90377def45/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:49.330820 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58c6257-2811-4c4e-92dd-71a6a8b13f4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=84258731610646279517fb4b69e87127, user_id=f364ea1c990a41e8a6aa6e90377def45, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:49.333637 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58c6257-2811-4c4e-92dd-71a6a8b13f4b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:49.361854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1373/2718] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:49 2022] PUT /identity/v3/projects/84258731610646279517fb4b69e87127/users/f364ea1c990a41e8a6aa6e90377def45/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:49.372924 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142f8e7-dc6b-40ec-b1c7-b60563a6d32c None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:49.373231 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142f8e7-dc6b-40ec-b1c7-b60563a6d32c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:49.373507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142f8e7-dc6b-40ec-b1c7-b60563a6d32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:49.425034 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-2142f8e7-dc6b-40ec-b1c7-b60563a6d32c None None] MFA Rules not processed for user `f364ea1c990a41e8a6aa6e90377def45`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:49.425895 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2142f8e7-dc6b-40ec-b1c7-b60563a6d32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:49.571667 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1346/2719] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:49.579110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c6257-2811-4c4e-92dd-71a6a8b13f4b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:49.581001 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e58c6257-2811-4c4e-92dd-71a6a8b13f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:49.685755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c6257-2811-4c4e-92dd-71a6a8b13f4b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:49.690988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662714a-50bf-49c3-9510-d972561f4364 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:49.691806 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662714a-50bf-49c3-9510-d972561f4364 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:49.692202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662714a-50bf-49c3-9510-d972561f4364 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:49.693727 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662714a-50bf-49c3-9510-d972561f4364 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:49.703794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662714a-50bf-49c3-9510-d972561f4364 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:49.705700 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662714a-50bf-49c3-9510-d972561f4364 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:49.716797 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1374/2720] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:52.027039 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142f8e7-dc6b-40ec-b1c7-b60563a6d32c None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:52.028541 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2142f8e7-dc6b-40ec-b1c7-b60563a6d32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:52.077497 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:52.122524 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142f8e7-dc6b-40ec-b1c7-b60563a6d32c None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:52.128339 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d927e-8275-4b9b-8511-9d7e205c2ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:52.129370 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d927e-8275-4b9b-8511-9d7e205c2ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:52.129962 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d927e-8275-4b9b-8511-9d7e205c2ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:52.130310 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d927e-8275-4b9b-8511-9d7e205c2ef3 admin admin] PATH_INFO: `/v3/users/74b2295521bc45509ef6b68c1885ac6e` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:52.131741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d927e-8275-4b9b-8511-9d7e205c2ef3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74b2295521bc45509ef6b68c1885ac6e)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:52.133751 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d927e-8275-4b9b-8511-9d7e205c2ef3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:52.190892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fa4d927e-8275-4b9b-8511-9d7e205c2ef3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74b2295521bc45509ef6b68c1885ac6e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:52.191548 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fa4d927e-8275-4b9b-8511-9d7e205c2ef3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74b2295521bc45509ef6b68c1885ac6e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:52.192241 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fa4d927e-8275-4b9b-8511-9d7e205c2ef3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74b2295521bc45509ef6b68c1885ac6e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:52.205422 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fa4d927e-8275-4b9b-8511-9d7e205c2ef3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74b2295521bc45509ef6b68c1885ac6e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:52.213434 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-fa4d927e-8275-4b9b-8511-9d7e205c2ef3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74b2295521bc45509ef6b68c1885ac6e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:52.242445 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1347/2721] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:52 2022] DELETE /identity/v3/users/74b2295521bc45509ef6b68c1885ac6e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:52.415069 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662714a-50bf-49c3-9510-d972561f4364 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:52.427934 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662714a-50bf-49c3-9510-d972561f4364 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:52.499391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:52.526617 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9ba56-cb84-4e90-aafb-8140d75a3b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:52.527511 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9ba56-cb84-4e90-aafb-8140d75a3b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:52.527823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9ba56-cb84-4e90-aafb-8140d75a3b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:52.528099 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9ba56-cb84-4e90-aafb-8140d75a3b23 admin admin] PATH_INFO: `/v3/projects/6b75fb163caa47cd9b72e047e9e050e2` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:52.529324 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9ba56-cb84-4e90-aafb-8140d75a3b23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b75fb163caa47cd9b72e047e9e050e2)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:52.531319 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9ba56-cb84-4e90-aafb-8140d75a3b23 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:52.591315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49f9ba56-cb84-4e90-aafb-8140d75a3b23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b75fb163caa47cd9b72e047e9e050e2', 'request_id': 'req-49f9ba56-cb84-4e90-aafb-8140d75a3b23'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:52.601579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49f9ba56-cb84-4e90-aafb-8140d75a3b23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b75fb163caa47cd9b72e047e9e050e2', 'request_id': 'req-49f9ba56-cb84-4e90-aafb-8140d75a3b23'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:52.608331 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49f9ba56-cb84-4e90-aafb-8140d75a3b23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b75fb163caa47cd9b72e047e9e050e2', 'request_id': 'req-49f9ba56-cb84-4e90-aafb-8140d75a3b23'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:52.620216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49f9ba56-cb84-4e90-aafb-8140d75a3b23 admin admin] The token cache is being invalidate because project 6b75fb163caa47cd9b72e047e9e050e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:52.620821 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49f9ba56-cb84-4e90-aafb-8140d75a3b23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:52.621481 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49f9ba56-cb84-4e90-aafb-8140d75a3b23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:52.623171 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1375/2722] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:52 2022] DELETE /identity/v3/projects/6b75fb163caa47cd9b72e047e9e050e2 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:52.635304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc711f2-ffbe-47a2-9659-fe34a2846c4c None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:52.635564 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc711f2-ffbe-47a2-9659-fe34a2846c4c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:52.635875 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc711f2-ffbe-47a2-9659-fe34a2846c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:52.656490 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ebc711f2-ffbe-47a2-9659-fe34a2846c4c None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:52.658665 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ebc711f2-ffbe-47a2-9659-fe34a2846c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:52.726456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1348/2723] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:52.734203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36969098-b1d0-450f-97ff-ef6aaac2f481 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:52.734497 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36969098-b1d0-450f-97ff-ef6aaac2f481 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:52.734778 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36969098-b1d0-450f-97ff-ef6aaac2f481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:52.755905 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-36969098-b1d0-450f-97ff-ef6aaac2f481 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:52.757157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-36969098-b1d0-450f-97ff-ef6aaac2f481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:52.765433 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1376/2724] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:52.770041 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc711f2-ffbe-47a2-9659-fe34a2846c4c None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:52.787309 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc711f2-ffbe-47a2-9659-fe34a2846c4c None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:52.792609 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39397118-cb45-4e2a-aab0-5f9bbf237df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:52.793286 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39397118-cb45-4e2a-aab0-5f9bbf237df6 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:52.793567 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39397118-cb45-4e2a-aab0-5f9bbf237df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:52.793837 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39397118-cb45-4e2a-aab0-5f9bbf237df6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:52.794599 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39397118-cb45-4e2a-aab0-5f9bbf237df6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:23:52.795179 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39397118-cb45-4e2a-aab0-5f9bbf237df6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:52.796960 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39397118-cb45-4e2a-aab0-5f9bbf237df6 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:52.822080 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1349/2725] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:23:52.823518 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36969098-b1d0-450f-97ff-ef6aaac2f481 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:52.836928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36969098-b1d0-450f-97ff-ef6aaac2f481 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:52.842389 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29c416-0b00-4c04-bb8b-d66ba1b10e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:52.843242 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29c416-0b00-4c04-bb8b-d66ba1b10e71 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:52.843860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29c416-0b00-4c04-bb8b-d66ba1b10e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:52.844338 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29c416-0b00-4c04-bb8b-d66ba1b10e71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:52.845200 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29c416-0b00-4c04-bb8b-d66ba1b10e71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:52.848357 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29c416-0b00-4c04-bb8b-d66ba1b10e71 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:52.897613 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1377/2726] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:52 2022] POST /identity/v3/projects => generated 375 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:52.901860 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39397118-cb45-4e2a-aab0-5f9bbf237df6 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:52.932694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39397118-cb45-4e2a-aab0-5f9bbf237df6 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:53.028378 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:53.055456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ba871-2dc8-40b0-849e-eaf8aa7d1b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:53.056226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ba871-2dc8-40b0-849e-eaf8aa7d1b42 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:53.056552 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ba871-2dc8-40b0-849e-eaf8aa7d1b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:53.056898 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ba871-2dc8-40b0-849e-eaf8aa7d1b42 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:53.057636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ba871-2dc8-40b0-849e-eaf8aa7d1b42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:53.059591 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ba871-2dc8-40b0-849e-eaf8aa7d1b42 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:53.091159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1350/2727] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:52 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:53.095342 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29c416-0b00-4c04-bb8b-d66ba1b10e71 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:53.107241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29c416-0b00-4c04-bb8b-d66ba1b10e71 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:53.111931 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef1760-1da2-46c1-9c6f-22a3fa54d948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:53.112608 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef1760-1da2-46c1-9c6f-22a3fa54d948 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:53.112890 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef1760-1da2-46c1-9c6f-22a3fa54d948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:53.113167 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef1760-1da2-46c1-9c6f-22a3fa54d948 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:53.113905 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef1760-1da2-46c1-9c6f-22a3fa54d948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:53.115516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef1760-1da2-46c1-9c6f-22a3fa54d948 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:53.127725 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1378/2728] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:23:53.131888 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ba871-2dc8-40b0-849e-eaf8aa7d1b42 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:53.143598 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ba871-2dc8-40b0-849e-eaf8aa7d1b42 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:53.148601 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c2039-823c-4a75-9817-e18027825e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:53.149465 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c2039-823c-4a75-9817-e18027825e94 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:53.149760 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c2039-823c-4a75-9817-e18027825e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:53.150033 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c2039-823c-4a75-9817-e18027825e94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:53.150769 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c2039-823c-4a75-9817-e18027825e94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:53.152429 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c2039-823c-4a75-9817-e18027825e94 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:53.164051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1351/2729] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:53.168405 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef1760-1da2-46c1-9c6f-22a3fa54d948 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:53.179152 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef1760-1da2-46c1-9c6f-22a3fa54d948 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:53.184150 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd8d2d-8483-4bc9-afc5-39ab091393f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:53.185075 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd8d2d-8483-4bc9-afc5-39ab091393f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:53.185359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd8d2d-8483-4bc9-afc5-39ab091393f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:53.185641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd8d2d-8483-4bc9-afc5-39ab091393f4 admin admin] PATH_INFO: `/v3/projects/412630dcb5ff4402983a02b249583995/users/6eb164ecd85f498f9c3b3aabe90dc054/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:53.205755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd8d2d-8483-4bc9-afc5-39ab091393f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=412630dcb5ff4402983a02b249583995, user_id=6eb164ecd85f498f9c3b3aabe90dc054, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:53.208948 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd8d2d-8483-4bc9-afc5-39ab091393f4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:53.226499 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1379/2730] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:53 2022] PUT /identity/v3/projects/412630dcb5ff4402983a02b249583995/users/6eb164ecd85f498f9c3b3aabe90dc054/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:53.232186 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8df14-dfce-48a3-9024-62afae862f62 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:53.232418 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8df14-dfce-48a3-9024-62afae862f62 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:53.232693 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8df14-dfce-48a3-9024-62afae862f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:53.249907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e1e8df14-dfce-48a3-9024-62afae862f62 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:53.251140 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e1e8df14-dfce-48a3-9024-62afae862f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:53.318141 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1352/2731] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:53.328867 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd8d2d-8483-4bc9-afc5-39ab091393f4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:53.330587 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c2fd8d2d-8483-4bc9-afc5-39ab091393f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:53.394124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd8d2d-8483-4bc9-afc5-39ab091393f4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:53.398999 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75ab28-4a8a-43d4-bc13-befb1a38d8ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:53.399988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75ab28-4a8a-43d4-bc13-befb1a38d8ed service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:53.400516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75ab28-4a8a-43d4-bc13-befb1a38d8ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:53.401024 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75ab28-4a8a-43d4-bc13-befb1a38d8ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:53.412799 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad75ab28-4a8a-43d4-bc13-befb1a38d8ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:53.414684 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad75ab28-4a8a-43d4-bc13-befb1a38d8ed service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:53.424697 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1380/2732] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:58.311190 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30024cab-7757-4df6-ba05-2035e199648f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:58.311818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30024cab-7757-4df6-ba05-2035e199648f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:58.311818 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30024cab-7757-4df6-ba05-2035e199648f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:58.343426 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-30024cab-7757-4df6-ba05-2035e199648f None None] MFA Rules not processed for user `6eb164ecd85f498f9c3b3aabe90dc054`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:58.345612 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-30024cab-7757-4df6-ba05-2035e199648f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:58.416210 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1353/2733] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:58.431002 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75ab28-4a8a-43d4-bc13-befb1a38d8ed service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:58.432536 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ad75ab28-4a8a-43d4-bc13-befb1a38d8ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:58.445588 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:58.512801 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75ab28-4a8a-43d4-bc13-befb1a38d8ed service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:58.518445 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd05c7b-8739-4dcf-8e89-877fa577d647 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:58.519114 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd05c7b-8739-4dcf-8e89-877fa577d647 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:58.519531 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd05c7b-8739-4dcf-8e89-877fa577d647 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:58.519845 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd05c7b-8739-4dcf-8e89-877fa577d647 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:58.530771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd05c7b-8739-4dcf-8e89-877fa577d647 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:58.532993 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd05c7b-8739-4dcf-8e89-877fa577d647 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:58.543059 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1381/2734] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:59.977688 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059582d-6d41-468f-a64a-8833e831b392 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:59.978247 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059582d-6d41-468f-a64a-8833e831b392 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:59.978695 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059582d-6d41-468f-a64a-8833e831b392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:00.011223 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e059582d-6d41-468f-a64a-8833e831b392 None None] MFA Rules not processed for user `6eb164ecd85f498f9c3b3aabe90dc054`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:00.012563 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e059582d-6d41-468f-a64a-8833e831b392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:00.020656 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1354/2735] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:23:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:08.538558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=30 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:10.005335 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:10.231737 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd05c7b-8739-4dcf-8e89-877fa577d647 service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:10.233341 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0bd05c7b-8739-4dcf-8e89-877fa577d647 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:10.246782 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd05c7b-8739-4dcf-8e89-877fa577d647 service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:10.252020 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055d06a-3ce4-491d-9159-e42c060d662d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:10.252927 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055d06a-3ce4-491d-9159-e42c060d662d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:10.253209 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055d06a-3ce4-491d-9159-e42c060d662d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:10.253468 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055d06a-3ce4-491d-9159-e42c060d662d admin admin] PATH_INFO: `/v3/users/f364ea1c990a41e8a6aa6e90377def45` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:10.254676 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055d06a-3ce4-491d-9159-e42c060d662d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f364ea1c990a41e8a6aa6e90377def45)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:10.256646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055d06a-3ce4-491d-9159-e42c060d662d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:10.306815 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9055d06a-3ce4-491d-9159-e42c060d662d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f364ea1c990a41e8a6aa6e90377def45'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:10.307499 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9055d06a-3ce4-491d-9159-e42c060d662d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f364ea1c990a41e8a6aa6e90377def45'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:10.308173 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9055d06a-3ce4-491d-9159-e42c060d662d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f364ea1c990a41e8a6aa6e90377def45'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:10.324471 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9055d06a-3ce4-491d-9159-e42c060d662d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f364ea1c990a41e8a6aa6e90377def45'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:10.336328 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9055d06a-3ce4-491d-9159-e42c060d662d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f364ea1c990a41e8a6aa6e90377def45'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:10.351704 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1382/2736] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:24:10 2022] DELETE /identity/v3/users/f364ea1c990a41e8a6aa6e90377def45 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:10.356180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c829c-48f2-439a-a6f5-ce94eb6da8c1 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:10.356475 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c829c-48f2-439a-a6f5-ce94eb6da8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:10.356755 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c829c-48f2-439a-a6f5-ce94eb6da8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:10.381913 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-8b8c829c-48f2-439a-a6f5-ce94eb6da8c1 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:10.383132 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8b8c829c-48f2-439a-a6f5-ce94eb6da8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:10.440921 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1355/2737] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:24:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:10.450654 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055d06a-3ce4-491d-9159-e42c060d662d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:10.452117 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9055d06a-3ce4-491d-9159-e42c060d662d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:10.511065 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055d06a-3ce4-491d-9159-e42c060d662d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:10.514881 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe61d25-d90a-463d-9c42-694309c4b368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:10.515545 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe61d25-d90a-463d-9c42-694309c4b368 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:10.515828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe61d25-d90a-463d-9c42-694309c4b368 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:10.516141 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe61d25-d90a-463d-9c42-694309c4b368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:10.525600 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe61d25-d90a-463d-9c42-694309c4b368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:10.527047 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe61d25-d90a-463d-9c42-694309c4b368 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:10.536595 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1383/2738] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:10.815056 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c829c-48f2-439a-a6f5-ce94eb6da8c1 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:10.816614 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8b8c829c-48f2-439a-a6f5-ce94eb6da8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:10.829202 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c829c-48f2-439a-a6f5-ce94eb6da8c1 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:10.836156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569a662-b7af-4b25-a47e-e997573c93bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:10.838107 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569a662-b7af-4b25-a47e-e997573c93bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:10.838464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569a662-b7af-4b25-a47e-e997573c93bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:10.839176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569a662-b7af-4b25-a47e-e997573c93bd admin admin] PATH_INFO: `/v3/projects/84258731610646279517fb4b69e87127` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:10.840970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569a662-b7af-4b25-a47e-e997573c93bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=84258731610646279517fb4b69e87127)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:10.844065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569a662-b7af-4b25-a47e-e997573c93bd admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:10.897335 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a569a662-b7af-4b25-a47e-e997573c93bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84258731610646279517fb4b69e87127', 'request_id': 'req-a569a662-b7af-4b25-a47e-e997573c93bd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:10.906190 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a569a662-b7af-4b25-a47e-e997573c93bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84258731610646279517fb4b69e87127', 'request_id': 'req-a569a662-b7af-4b25-a47e-e997573c93bd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:10.912026 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a569a662-b7af-4b25-a47e-e997573c93bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84258731610646279517fb4b69e87127', 'request_id': 'req-a569a662-b7af-4b25-a47e-e997573c93bd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:10.922869 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a569a662-b7af-4b25-a47e-e997573c93bd admin admin] The token cache is being invalidate because project 84258731610646279517fb4b69e87127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:24:10.923156 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a569a662-b7af-4b25-a47e-e997573c93bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:10.923804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a569a662-b7af-4b25-a47e-e997573c93bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:10.925335 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1356/2739] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:24:10 2022] DELETE /identity/v3/projects/84258731610646279517fb4b69e87127 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:16.574695 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe61d25-d90a-463d-9c42-694309c4b368 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:16.584243 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe61d25-d90a-463d-9c42-694309c4b368 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:16.638310 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4631f6-dc30-478d-8375-98768521dcb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:16.638961 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4631f6-dc30-478d-8375-98768521dcb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:16.639240 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4631f6-dc30-478d-8375-98768521dcb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:16.639511 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4631f6-dc30-478d-8375-98768521dcb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:16.641441 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3d4631f6-dc30-478d-8375-98768521dcb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:16.677386 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:16.721652 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4631f6-dc30-478d-8375-98768521dcb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:16.723136 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4631f6-dc30-478d-8375-98768521dcb0 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:16.733113 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1384/2740] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:16 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:20.919989 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:21.404002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569a662-b7af-4b25-a47e-e997573c93bd admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:21.405556 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a569a662-b7af-4b25-a47e-e997573c93bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:21.418300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569a662-b7af-4b25-a47e-e997573c93bd admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:21.422376 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc4083-8d34-435b-8267-9ab812354442 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:21.423015 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc4083-8d34-435b-8267-9ab812354442 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:21.423289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc4083-8d34-435b-8267-9ab812354442 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:21.423584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc4083-8d34-435b-8267-9ab812354442 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:21.425454 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-75fc4083-8d34-435b-8267-9ab812354442 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:21.493458 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc4083-8d34-435b-8267-9ab812354442 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:21.494919 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc4083-8d34-435b-8267-9ab812354442 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:21.505093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1357/2741] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:21.900920 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4631f6-dc30-478d-8375-98768521dcb0 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:21.902453 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3d4631f6-dc30-478d-8375-98768521dcb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:21.986646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4631f6-dc30-478d-8375-98768521dcb0 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:21.992304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ec883-473f-41a3-aaa1-48692e2d4673 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5fa9b0b0314ee9bb7e16b8464307b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:21.993003 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ec883-473f-41a3-aaa1-48692e2d4673 service placement] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:21.993298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ec883-473f-41a3-aaa1-48692e2d4673 service placement] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:21.993696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ec883-473f-41a3-aaa1-48692e2d4673 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:21.995632 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-930ec883-473f-41a3-aaa1-48692e2d4673 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:22.010533 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ec883-473f-41a3-aaa1-48692e2d4673 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:22.012244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ec883-473f-41a3-aaa1-48692e2d4673 service placement] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:22.023352 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1385/2742] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:23.082645 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc4083-8d34-435b-8267-9ab812354442 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:23.093408 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc4083-8d34-435b-8267-9ab812354442 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:23.097572 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74239f9f-9777-4ef3-ab5c-fc37067bca57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:23.098397 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74239f9f-9777-4ef3-ab5c-fc37067bca57 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:23.099117 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74239f9f-9777-4ef3-ab5c-fc37067bca57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:23.099117 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74239f9f-9777-4ef3-ab5c-fc37067bca57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:23.102791 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-74239f9f-9777-4ef3-ab5c-fc37067bca57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:23.192641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74239f9f-9777-4ef3-ab5c-fc37067bca57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:23.194139 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74239f9f-9777-4ef3-ab5c-fc37067bca57 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:23.204743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1358/2743] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:30.545919 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ec883-473f-41a3-aaa1-48692e2d4673 service placement] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:30.547560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-930ec883-473f-41a3-aaa1-48692e2d4673 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:30.562295 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ec883-473f-41a3-aaa1-48692e2d4673 service placement] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:30.566739 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7f68e-91f5-44a0-978d-a51ba23dbe4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:30.567648 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7f68e-91f5-44a0-978d-a51ba23dbe4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:30.567939 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7f68e-91f5-44a0-978d-a51ba23dbe4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:30.568215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7f68e-91f5-44a0-978d-a51ba23dbe4e admin admin] PATH_INFO: `/v3/users/4f9b08c0c7fd4d2e957afcefca500751` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:30.569449 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7f68e-91f5-44a0-978d-a51ba23dbe4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f9b08c0c7fd4d2e957afcefca500751)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:30.571371 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7f68e-91f5-44a0-978d-a51ba23dbe4e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:30.620036 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e9a7f68e-91f5-44a0-978d-a51ba23dbe4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f9b08c0c7fd4d2e957afcefca500751'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:30.620703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e9a7f68e-91f5-44a0-978d-a51ba23dbe4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f9b08c0c7fd4d2e957afcefca500751'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:30.621317 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e9a7f68e-91f5-44a0-978d-a51ba23dbe4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f9b08c0c7fd4d2e957afcefca500751'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:30.633489 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e9a7f68e-91f5-44a0-978d-a51ba23dbe4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f9b08c0c7fd4d2e957afcefca500751'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:30.640433 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e9a7f68e-91f5-44a0-978d-a51ba23dbe4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f9b08c0c7fd4d2e957afcefca500751'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:30.654013 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1386/2744] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:24:30 2022] DELETE /identity/v3/users/4f9b08c0c7fd4d2e957afcefca500751 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:30.848042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74239f9f-9777-4ef3-ab5c-fc37067bca57 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:30.861959 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74239f9f-9777-4ef3-ab5c-fc37067bca57 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:30.928048 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009e2a0-ff67-4a24-b08b-8287f6675915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:30.928929 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009e2a0-ff67-4a24-b08b-8287f6675915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:30.929207 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009e2a0-ff67-4a24-b08b-8287f6675915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:30.929478 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009e2a0-ff67-4a24-b08b-8287f6675915 admin admin] PATH_INFO: `/v3/projects/17dea7e9f26c4b74a10d5e00a395fbd7` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:30.930705 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009e2a0-ff67-4a24-b08b-8287f6675915 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17dea7e9f26c4b74a10d5e00a395fbd7)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:30.932726 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009e2a0-ff67-4a24-b08b-8287f6675915 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:30.969875 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:30.985894 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b009e2a0-ff67-4a24-b08b-8287f6675915 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17dea7e9f26c4b74a10d5e00a395fbd7', 'request_id': 'req-b009e2a0-ff67-4a24-b08b-8287f6675915'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:30.998873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b009e2a0-ff67-4a24-b08b-8287f6675915 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17dea7e9f26c4b74a10d5e00a395fbd7', 'request_id': 'req-b009e2a0-ff67-4a24-b08b-8287f6675915'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:31.004918 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b009e2a0-ff67-4a24-b08b-8287f6675915 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17dea7e9f26c4b74a10d5e00a395fbd7', 'request_id': 'req-b009e2a0-ff67-4a24-b08b-8287f6675915'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:31.018004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b009e2a0-ff67-4a24-b08b-8287f6675915 admin admin] The token cache is being invalidate because project 17dea7e9f26c4b74a10d5e00a395fbd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:24:31.018292 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b009e2a0-ff67-4a24-b08b-8287f6675915 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:31.018983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b009e2a0-ff67-4a24-b08b-8287f6675915 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:31.020617 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1359/2745] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:24:30 2022] DELETE /identity/v3/projects/17dea7e9f26c4b74a10d5e00a395fbd7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:31.024756 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7f68e-91f5-44a0-978d-a51ba23dbe4e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:31.026160 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e9a7f68e-91f5-44a0-978d-a51ba23dbe4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:31.039681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7f68e-91f5-44a0-978d-a51ba23dbe4e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:31.068814 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:31.098734 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2749a61-7754-4d34-854d-2cc8c283cc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:31.099701 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2749a61-7754-4d34-854d-2cc8c283cc91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:31.099985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2749a61-7754-4d34-854d-2cc8c283cc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:31.100261 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2749a61-7754-4d34-854d-2cc8c283cc91 admin admin] PATH_INFO: `/v3/users/eb045f8f97304e9387d29ea6f6b20ef3` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:31.101564 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2749a61-7754-4d34-854d-2cc8c283cc91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb045f8f97304e9387d29ea6f6b20ef3)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:31.103549 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2749a61-7754-4d34-854d-2cc8c283cc91 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:31.156740 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c2749a61-7754-4d34-854d-2cc8c283cc91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb045f8f97304e9387d29ea6f6b20ef3'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:31.157488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c2749a61-7754-4d34-854d-2cc8c283cc91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb045f8f97304e9387d29ea6f6b20ef3'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:31.158263 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c2749a61-7754-4d34-854d-2cc8c283cc91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb045f8f97304e9387d29ea6f6b20ef3'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:31.173770 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c2749a61-7754-4d34-854d-2cc8c283cc91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb045f8f97304e9387d29ea6f6b20ef3'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:31.181894 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c2749a61-7754-4d34-854d-2cc8c283cc91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb045f8f97304e9387d29ea6f6b20ef3'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:31.196846 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1387/2746] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:24:31 2022] DELETE /identity/v3/users/eb045f8f97304e9387d29ea6f6b20ef3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:31.378746 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009e2a0-ff67-4a24-b08b-8287f6675915 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:31.380262 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b009e2a0-ff67-4a24-b08b-8287f6675915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:31.394854 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009e2a0-ff67-4a24-b08b-8287f6675915 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:31.453029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2eace-0e74-4d7d-94c8-4d6f71e81b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:31.453894 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2eace-0e74-4d7d-94c8-4d6f71e81b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:31.454250 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2eace-0e74-4d7d-94c8-4d6f71e81b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:31.454525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2eace-0e74-4d7d-94c8-4d6f71e81b81 admin admin] PATH_INFO: `/v3/projects/4370c77bae0e4c4f9f6772a7d66cdab3` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:31.455727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2eace-0e74-4d7d-94c8-4d6f71e81b81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4370c77bae0e4c4f9f6772a7d66cdab3)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:31.457642 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2eace-0e74-4d7d-94c8-4d6f71e81b81 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:31.511536 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a4c2eace-0e74-4d7d-94c8-4d6f71e81b81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4370c77bae0e4c4f9f6772a7d66cdab3', 'request_id': 'req-a4c2eace-0e74-4d7d-94c8-4d6f71e81b81'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:31.519889 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a4c2eace-0e74-4d7d-94c8-4d6f71e81b81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4370c77bae0e4c4f9f6772a7d66cdab3', 'request_id': 'req-a4c2eace-0e74-4d7d-94c8-4d6f71e81b81'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:31.527911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a4c2eace-0e74-4d7d-94c8-4d6f71e81b81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4370c77bae0e4c4f9f6772a7d66cdab3', 'request_id': 'req-a4c2eace-0e74-4d7d-94c8-4d6f71e81b81'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:31.539428 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a4c2eace-0e74-4d7d-94c8-4d6f71e81b81 admin admin] The token cache is being invalidate because project 4370c77bae0e4c4f9f6772a7d66cdab3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:24:31.539747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a4c2eace-0e74-4d7d-94c8-4d6f71e81b81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:31.540378 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a4c2eace-0e74-4d7d-94c8-4d6f71e81b81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:31.541889 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1360/2747] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:24:31 2022] DELETE /identity/v3/projects/4370c77bae0e4c4f9f6772a7d66cdab3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:41.193551 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=39,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:41.536210 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=64,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:47.599543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2749a61-7754-4d34-854d-2cc8c283cc91 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:47.602189 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c2749a61-7754-4d34-854d-2cc8c283cc91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:47.666506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2749a61-7754-4d34-854d-2cc8c283cc91 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:47.716932 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31452766-1853-4f1e-8985-85c785a817ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:47.717747 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31452766-1853-4f1e-8985-85c785a817ef service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:47.718032 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31452766-1853-4f1e-8985-85c785a817ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:47.718303 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31452766-1853-4f1e-8985-85c785a817ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:47.720303 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-31452766-1853-4f1e-8985-85c785a817ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:47.780461 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31452766-1853-4f1e-8985-85c785a817ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:47.781912 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31452766-1853-4f1e-8985-85c785a817ef service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:47.791539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1388/2748] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:55.697940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61794edb-91b4-4579-be61-d1c5a3cd1b12 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:55.698427 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61794edb-91b4-4579-be61-d1c5a3cd1b12 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:55.698577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61794edb-91b4-4579-be61-d1c5a3cd1b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:55.720456 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-61794edb-91b4-4579-be61-d1c5a3cd1b12 None None] MFA Rules not processed for user `32b9a1c24fe84019a311b1331ea2c004`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:55.721704 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-61794edb-91b4-4579-be61-d1c5a3cd1b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:55.729318 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1361/2749] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:24:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:57.787264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=68 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:25:05.712813 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:25:16.767571 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31452766-1853-4f1e-8985-85c785a817ef service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:25:16.769106 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-31452766-1853-4f1e-8985-85c785a817ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:25:16.826372 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31452766-1853-4f1e-8985-85c785a817ef service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:25:16.830367 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982474f5-72fb-4b29-87e9-571a65210671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:25:16.831022 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982474f5-72fb-4b29-87e9-571a65210671 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:25:16.831297 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982474f5-72fb-4b29-87e9-571a65210671 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:25:16.831565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982474f5-72fb-4b29-87e9-571a65210671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:25:16.833565 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-982474f5-72fb-4b29-87e9-571a65210671 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:25:16.899085 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982474f5-72fb-4b29-87e9-571a65210671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:25:16.900789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982474f5-72fb-4b29-87e9-571a65210671 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:25:16.911631 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1389/2750] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:25:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:25:26.907484 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=50 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:25:49.857306 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6847a-8cc2-4d78-b6b2-1c683051be7f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:25:49.857777 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6847a-8cc2-4d78-b6b2-1c683051be7f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:25:49.857872 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6847a-8cc2-4d78-b6b2-1c683051be7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:25:49.876870 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-79a6847a-8cc2-4d78-b6b2-1c683051be7f None None] MFA Rules not processed for user `e802da0b078e457da466a9a7b0f15085`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:25:49.878065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-79a6847a-8cc2-4d78-b6b2-1c683051be7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:25:49.939150 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1362/2751] 10.222.0.27 () {58 vars in 967 bytes} [Mon Jun 27 21:25:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:25:49.988135 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982474f5-72fb-4b29-87e9-571a65210671 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:25:49.989754 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-982474f5-72fb-4b29-87e9-571a65210671 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:25:50.060905 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982474f5-72fb-4b29-87e9-571a65210671 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:25:50.066817 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e354f08-4885-4183-9634-9165a86fd226 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28acb0322114aebafcf62bdf9a346df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:25:50.067512 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e354f08-4885-4183-9634-9165a86fd226 service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:25:50.067827 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e354f08-4885-4183-9634-9165a86fd226 service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:25:50.068111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e354f08-4885-4183-9634-9165a86fd226 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:25:50.077298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e354f08-4885-4183-9634-9165a86fd226 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:25:50.078878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e354f08-4885-4183-9634-9165a86fd226 service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:25:50.087526 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1390/2752] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Jun 27 21:25:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:25:56.288830 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6847a-8cc2-4d78-b6b2-1c683051be7f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:25:56.290534 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-79a6847a-8cc2-4d78-b6b2-1c683051be7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:25:56.363016 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6847a-8cc2-4d78-b6b2-1c683051be7f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:25:56.367583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5982a-e49f-4d63-a629-a2fa904a8b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:25:56.368522 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5982a-e49f-4d63-a629-a2fa904a8b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:25:56.368805 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5982a-e49f-4d63-a629-a2fa904a8b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:25:56.369079 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5982a-e49f-4d63-a629-a2fa904a8b02 admin admin] PATH_INFO: `/v3/users/19e54ac48c1943b0999baeaad67a0e11` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:25:56.370305 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5982a-e49f-4d63-a629-a2fa904a8b02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19e54ac48c1943b0999baeaad67a0e11)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:25:56.372725 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5982a-e49f-4d63-a629-a2fa904a8b02 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:25:56.422503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-43b5982a-e49f-4d63-a629-a2fa904a8b02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19e54ac48c1943b0999baeaad67a0e11'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:25:56.423166 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-43b5982a-e49f-4d63-a629-a2fa904a8b02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19e54ac48c1943b0999baeaad67a0e11'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:25:56.423798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-43b5982a-e49f-4d63-a629-a2fa904a8b02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19e54ac48c1943b0999baeaad67a0e11'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:25:56.435352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-43b5982a-e49f-4d63-a629-a2fa904a8b02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19e54ac48c1943b0999baeaad67a0e11'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:25:56.442846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-43b5982a-e49f-4d63-a629-a2fa904a8b02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19e54ac48c1943b0999baeaad67a0e11'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:25:56.458284 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1363/2753] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:25:56 2022] DELETE /identity/v3/users/19e54ac48c1943b0999baeaad67a0e11 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:25:56.641129 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e354f08-4885-4183-9634-9165a86fd226 service swift] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:25:56.656549 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e354f08-4885-4183-9634-9165a86fd226 service swift] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:25:56.714331 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963c8f8-a7b6-4e2d-9b0a-af0841f1668b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:25:56.715207 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963c8f8-a7b6-4e2d-9b0a-af0841f1668b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:25:56.715487 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963c8f8-a7b6-4e2d-9b0a-af0841f1668b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:25:56.715795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963c8f8-a7b6-4e2d-9b0a-af0841f1668b admin admin] PATH_INFO: `/v3/projects/61525140325a4b14aa10df2b64efa5cd` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:25:56.717004 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963c8f8-a7b6-4e2d-9b0a-af0841f1668b admin admin] RBAC: Authorizing `identity:delete_project(project_id=61525140325a4b14aa10df2b64efa5cd)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:25:56.718927 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963c8f8-a7b6-4e2d-9b0a-af0841f1668b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:25:56.775263 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b963c8f8-a7b6-4e2d-9b0a-af0841f1668b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61525140325a4b14aa10df2b64efa5cd', 'request_id': 'req-b963c8f8-a7b6-4e2d-9b0a-af0841f1668b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:25:56.783413 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b963c8f8-a7b6-4e2d-9b0a-af0841f1668b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61525140325a4b14aa10df2b64efa5cd', 'request_id': 'req-b963c8f8-a7b6-4e2d-9b0a-af0841f1668b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:25:56.789008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b963c8f8-a7b6-4e2d-9b0a-af0841f1668b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61525140325a4b14aa10df2b64efa5cd', 'request_id': 'req-b963c8f8-a7b6-4e2d-9b0a-af0841f1668b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:25:56.800198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b963c8f8-a7b6-4e2d-9b0a-af0841f1668b admin admin] The token cache is being invalidate because project 61525140325a4b14aa10df2b64efa5cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:25:56.800504 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b963c8f8-a7b6-4e2d-9b0a-af0841f1668b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:25:56.801126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b963c8f8-a7b6-4e2d-9b0a-af0841f1668b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:25:56.802714 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1391/2754] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:25:56 2022] DELETE /identity/v3/projects/61525140325a4b14aa10df2b64efa5cd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:25:56.806892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5982a-e49f-4d63-a629-a2fa904a8b02 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:25:56.808410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-43b5982a-e49f-4d63-a629-a2fa904a8b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:25:56.822121 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5982a-e49f-4d63-a629-a2fa904a8b02 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:25:56.842985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:25:56.884195 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3f88c-b7c9-4d23-ad5a-9bd572d4ca54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:25:56.885093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3f88c-b7c9-4d23-ad5a-9bd572d4ca54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:25:56.885365 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3f88c-b7c9-4d23-ad5a-9bd572d4ca54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:25:56.885635 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3f88c-b7c9-4d23-ad5a-9bd572d4ca54 admin admin] PATH_INFO: `/v3/users/d7eda7675d7648c3aff7e869a5718358` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:25:56.886853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3f88c-b7c9-4d23-ad5a-9bd572d4ca54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7eda7675d7648c3aff7e869a5718358)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:25:56.889005 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3f88c-b7c9-4d23-ad5a-9bd572d4ca54 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:25:56.939264 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-cdb3f88c-b7c9-4d23-ad5a-9bd572d4ca54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7eda7675d7648c3aff7e869a5718358'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:25:56.939953 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-cdb3f88c-b7c9-4d23-ad5a-9bd572d4ca54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7eda7675d7648c3aff7e869a5718358'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:25:56.940598 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-cdb3f88c-b7c9-4d23-ad5a-9bd572d4ca54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7eda7675d7648c3aff7e869a5718358'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:25:56.954164 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-cdb3f88c-b7c9-4d23-ad5a-9bd572d4ca54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7eda7675d7648c3aff7e869a5718358'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:25:56.961853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-cdb3f88c-b7c9-4d23-ad5a-9bd572d4ca54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7eda7675d7648c3aff7e869a5718358'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:25:56.986076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1364/2755] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:25:56 2022] DELETE /identity/v3/users/d7eda7675d7648c3aff7e869a5718358 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:25:57.426850 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963c8f8-a7b6-4e2d-9b0a-af0841f1668b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:25:57.428582 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b963c8f8-a7b6-4e2d-9b0a-af0841f1668b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:25:57.442202 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963c8f8-a7b6-4e2d-9b0a-af0841f1668b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:25:57.462991 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:25:57.500394 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3f88c-b7c9-4d23-ad5a-9bd572d4ca54 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:25:57.501775 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-cdb3f88c-b7c9-4d23-ad5a-9bd572d4ca54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:25:57.506882 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652bc11-c470-447d-bed4-a1cc2ad6eac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:25:57.507795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652bc11-c470-447d-bed4-a1cc2ad6eac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:25:57.508073 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652bc11-c470-447d-bed4-a1cc2ad6eac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:25:57.508346 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652bc11-c470-447d-bed4-a1cc2ad6eac6 admin admin] PATH_INFO: `/v3/projects/aa995a6e9663423bb9079d9126f92f0f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:25:57.509514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652bc11-c470-447d-bed4-a1cc2ad6eac6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa995a6e9663423bb9079d9126f92f0f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:25:57.511442 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652bc11-c470-447d-bed4-a1cc2ad6eac6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:25:57.564070 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9652bc11-c470-447d-bed4-a1cc2ad6eac6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa995a6e9663423bb9079d9126f92f0f', 'request_id': 'req-9652bc11-c470-447d-bed4-a1cc2ad6eac6'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:25:57.566694 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3f88c-b7c9-4d23-ad5a-9bd572d4ca54 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:25:57.571966 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9652bc11-c470-447d-bed4-a1cc2ad6eac6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa995a6e9663423bb9079d9126f92f0f', 'request_id': 'req-9652bc11-c470-447d-bed4-a1cc2ad6eac6'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:25:57.578215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9652bc11-c470-447d-bed4-a1cc2ad6eac6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa995a6e9663423bb9079d9126f92f0f', 'request_id': 'req-9652bc11-c470-447d-bed4-a1cc2ad6eac6'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:25:57.589781 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9652bc11-c470-447d-bed4-a1cc2ad6eac6 admin admin] The token cache is being invalidate because project aa995a6e9663423bb9079d9126f92f0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:25:57.590001 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9652bc11-c470-447d-bed4-a1cc2ad6eac6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:25:57.590738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9652bc11-c470-447d-bed4-a1cc2ad6eac6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:25:57.592373 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1392/2756] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:25:57 2022] DELETE /identity/v3/projects/aa995a6e9663423bb9079d9126f92f0f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:25:57.622370 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48762450-5b54-4500-970f-6ff92b36a7e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:25:57.623051 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48762450-5b54-4500-970f-6ff92b36a7e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:25:57.623337 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48762450-5b54-4500-970f-6ff92b36a7e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:25:57.623648 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48762450-5b54-4500-970f-6ff92b36a7e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:25:57.625613 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-48762450-5b54-4500-970f-6ff92b36a7e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:25:57.641062 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:25:57.711356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48762450-5b54-4500-970f-6ff92b36a7e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:25:57.712879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48762450-5b54-4500-970f-6ff92b36a7e8 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:25:57.725523 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1365/2757] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:25:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:26:00.307383 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652bc11-c470-447d-bed4-a1cc2ad6eac6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:26:00.308736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9652bc11-c470-447d-bed4-a1cc2ad6eac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:00.374312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652bc11-c470-447d-bed4-a1cc2ad6eac6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:26:00.378514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2beaf2-3be3-4764-8635-b8c880026240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:26:00.379154 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2beaf2-3be3-4764-8635-b8c880026240 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:26:00.379436 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2beaf2-3be3-4764-8635-b8c880026240 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:26:00.379745 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2beaf2-3be3-4764-8635-b8c880026240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:26:00.381707 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9e2beaf2-3be3-4764-8635-b8c880026240 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:00.396107 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2beaf2-3be3-4764-8635-b8c880026240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:26:00.397538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2beaf2-3be3-4764-8635-b8c880026240 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:26:00.408431 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1393/2758] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:26:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:26:07.722218 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=27 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:26:10.403630 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=68,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:26:49.128381 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48762450-5b54-4500-970f-6ff92b36a7e8 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:26:49.130074 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-48762450-5b54-4500-970f-6ff92b36a7e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:49.145799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48762450-5b54-4500-970f-6ff92b36a7e8 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:26:49.153045 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3f357-21ed-4a56-a4d3-ffe0eae07c04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:26:49.153722 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3f357-21ed-4a56-a4d3-ffe0eae07c04 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:26:49.154004 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3f357-21ed-4a56-a4d3-ffe0eae07c04 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:26:49.154280 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3f357-21ed-4a56-a4d3-ffe0eae07c04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:26:49.156576 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2de3f357-21ed-4a56-a4d3-ffe0eae07c04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:49.171547 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3f357-21ed-4a56-a4d3-ffe0eae07c04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:26:49.173285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3f357-21ed-4a56-a4d3-ffe0eae07c04 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:26:49.195823 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1366/2759] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:26:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:26:52.570381 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae094a74-fba7-4db8-8531-0f84b67dc73c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:26:52.570913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae094a74-fba7-4db8-8531-0f84b67dc73c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:26:52.570913 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae094a74-fba7-4db8-8531-0f84b67dc73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:26:52.588440 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ae094a74-fba7-4db8-8531-0f84b67dc73c None None] MFA Rules not processed for user `6eb164ecd85f498f9c3b3aabe90dc054`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:26:52.589665 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ae094a74-fba7-4db8-8531-0f84b67dc73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:52.646024 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1394/2760] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:26:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:26:53.288929 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3f357-21ed-4a56-a4d3-ffe0eae07c04 service nova] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:26:53.290563 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2de3f357-21ed-4a56-a4d3-ffe0eae07c04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:53.380748 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3f357-21ed-4a56-a4d3-ffe0eae07c04 service nova] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:26:53.386257 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade8caa-ca93-4f21-a529-8307075e2d25 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5fa9b0b0314ee9bb7e16b8464307b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:26:53.386938 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade8caa-ca93-4f21-a529-8307075e2d25 service placement] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:26:53.387222 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade8caa-ca93-4f21-a529-8307075e2d25 service placement] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:26:53.387495 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade8caa-ca93-4f21-a529-8307075e2d25 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:26:53.389410 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3ade8caa-ca93-4f21-a529-8307075e2d25 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:53.401046 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade8caa-ca93-4f21-a529-8307075e2d25 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:26:53.402641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade8caa-ca93-4f21-a529-8307075e2d25 service placement] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:26:53.413070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1367/2761] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:26:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:26:55.415718 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae094a74-fba7-4db8-8531-0f84b67dc73c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:26:55.426232 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae094a74-fba7-4db8-8531-0f84b67dc73c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:26:55.430063 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f462a-6a4d-49d1-bee9-df39780a5f16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:26:55.430738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f462a-6a4d-49d1-bee9-df39780a5f16 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:26:55.431016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f462a-6a4d-49d1-bee9-df39780a5f16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:26:55.431284 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f462a-6a4d-49d1-bee9-df39780a5f16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:26:55.433181 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-547f462a-6a4d-49d1-bee9-df39780a5f16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:55.444143 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f462a-6a4d-49d1-bee9-df39780a5f16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:26:55.445774 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f462a-6a4d-49d1-bee9-df39780a5f16 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:26:55.456083 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1395/2762] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:26:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:27:01.428793 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade8caa-ca93-4f21-a529-8307075e2d25 service placement] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:27:01.443198 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade8caa-ca93-4f21-a529-8307075e2d25 service placement] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:27:01.450089 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98142fde-557b-480f-966b-17c03a05f163 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:27:01.450407 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98142fde-557b-480f-966b-17c03a05f163 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:27:01.450693 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98142fde-557b-480f-966b-17c03a05f163 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:27:01.450968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98142fde-557b-480f-966b-17c03a05f163 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:27:01.453072 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-98142fde-557b-480f-966b-17c03a05f163 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:27:01.467827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98142fde-557b-480f-966b-17c03a05f163 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:27:01.469521 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98142fde-557b-480f-966b-17c03a05f163 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:27:01.481770 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1368/2763] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:27:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:27:05.451115 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=31 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:27:11.391229 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f462a-6a4d-49d1-bee9-df39780a5f16 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:27:11.400972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f462a-6a4d-49d1-bee9-df39780a5f16 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:27:11.405198 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a051d66-8801-4c61-91c3-0be56de669ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:27:11.405873 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a051d66-8801-4c61-91c3-0be56de669ac service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:27:11.406162 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a051d66-8801-4c61-91c3-0be56de669ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:27:11.406460 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a051d66-8801-4c61-91c3-0be56de669ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:27:11.408711 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4a051d66-8801-4c61-91c3-0be56de669ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:27:11.475861 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a051d66-8801-4c61-91c3-0be56de669ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:27:11.476309 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=44 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:27:11.477366 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a051d66-8801-4c61-91c3-0be56de669ac service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:27:11.489496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1396/2764] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:27:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:27:17.962262 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98142fde-557b-480f-966b-17c03a05f163 service nova] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:27:17.963850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-98142fde-557b-480f-966b-17c03a05f163 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:27:17.977775 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98142fde-557b-480f-966b-17c03a05f163 service nova] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:27:17.982538 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e0696-9f2e-4fe3-82f5-54a4ee3002e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:27:17.983427 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e0696-9f2e-4fe3-82f5-54a4ee3002e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:27:17.983736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e0696-9f2e-4fe3-82f5-54a4ee3002e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:27:17.984011 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e0696-9f2e-4fe3-82f5-54a4ee3002e3 admin admin] PATH_INFO: `/v3/users/32b9a1c24fe84019a311b1331ea2c004` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:27:17.985259 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e0696-9f2e-4fe3-82f5-54a4ee3002e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32b9a1c24fe84019a311b1331ea2c004)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:27:17.987353 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e0696-9f2e-4fe3-82f5-54a4ee3002e3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:27:18.039259 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2a9e0696-9f2e-4fe3-82f5-54a4ee3002e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32b9a1c24fe84019a311b1331ea2c004'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:27:18.040059 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2a9e0696-9f2e-4fe3-82f5-54a4ee3002e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32b9a1c24fe84019a311b1331ea2c004'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:27:18.040706 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2a9e0696-9f2e-4fe3-82f5-54a4ee3002e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32b9a1c24fe84019a311b1331ea2c004'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:27:18.052417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2a9e0696-9f2e-4fe3-82f5-54a4ee3002e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32b9a1c24fe84019a311b1331ea2c004'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:27:18.059507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2a9e0696-9f2e-4fe3-82f5-54a4ee3002e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32b9a1c24fe84019a311b1331ea2c004'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:27:18.075899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1369/2765] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:27:17 2022] DELETE /identity/v3/users/32b9a1c24fe84019a311b1331ea2c004 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:27:18.271289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a051d66-8801-4c61-91c3-0be56de669ac service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:27:18.281783 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a051d66-8801-4c61-91c3-0be56de669ac service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:27:18.339761 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2cd72-72b1-4323-b514-a80f2eb26161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:27:18.340717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2cd72-72b1-4323-b514-a80f2eb26161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:27:18.340972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2cd72-72b1-4323-b514-a80f2eb26161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:27:18.341265 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2cd72-72b1-4323-b514-a80f2eb26161 admin admin] PATH_INFO: `/v3/projects/4e726ed669c64a9b9d618886f1cc96de` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:27:18.342508 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2cd72-72b1-4323-b514-a80f2eb26161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e726ed669c64a9b9d618886f1cc96de)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:27:18.344482 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2cd72-72b1-4323-b514-a80f2eb26161 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:27:18.392933 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-58c2cd72-72b1-4323-b514-a80f2eb26161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e726ed669c64a9b9d618886f1cc96de', 'request_id': 'req-58c2cd72-72b1-4323-b514-a80f2eb26161'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:27:18.400385 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-58c2cd72-72b1-4323-b514-a80f2eb26161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e726ed669c64a9b9d618886f1cc96de', 'request_id': 'req-58c2cd72-72b1-4323-b514-a80f2eb26161'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:27:18.406113 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-58c2cd72-72b1-4323-b514-a80f2eb26161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e726ed669c64a9b9d618886f1cc96de', 'request_id': 'req-58c2cd72-72b1-4323-b514-a80f2eb26161'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:27:18.417148 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-58c2cd72-72b1-4323-b514-a80f2eb26161 admin admin] The token cache is being invalidate because project 4e726ed669c64a9b9d618886f1cc96de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:27:18.417437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-58c2cd72-72b1-4323-b514-a80f2eb26161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:27:18.418174 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-58c2cd72-72b1-4323-b514-a80f2eb26161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:27:18.419851 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1397/2766] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:27:18 2022] DELETE /identity/v3/projects/4e726ed669c64a9b9d618886f1cc96de => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:27:18.424070 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e0696-9f2e-4fe3-82f5-54a4ee3002e3 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:27:18.425570 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2a9e0696-9f2e-4fe3-82f5-54a4ee3002e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:27:18.439566 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e0696-9f2e-4fe3-82f5-54a4ee3002e3 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:27:18.498501 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c204dfa-fdee-4689-a68d-b06a3b06e681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:27:18.499299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c204dfa-fdee-4689-a68d-b06a3b06e681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:27:18.499577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c204dfa-fdee-4689-a68d-b06a3b06e681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:27:18.499906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c204dfa-fdee-4689-a68d-b06a3b06e681 admin admin] PATH_INFO: `/v3/users/0e8a1b34b723424e8552493a6c582aa6` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:27:18.501466 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c204dfa-fdee-4689-a68d-b06a3b06e681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e8a1b34b723424e8552493a6c582aa6)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:27:18.503415 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c204dfa-fdee-4689-a68d-b06a3b06e681 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:27:18.550232 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6c204dfa-fdee-4689-a68d-b06a3b06e681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e8a1b34b723424e8552493a6c582aa6'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:27:18.550875 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6c204dfa-fdee-4689-a68d-b06a3b06e681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e8a1b34b723424e8552493a6c582aa6'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:27:18.551467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6c204dfa-fdee-4689-a68d-b06a3b06e681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e8a1b34b723424e8552493a6c582aa6'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:27:18.563154 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6c204dfa-fdee-4689-a68d-b06a3b06e681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e8a1b34b723424e8552493a6c582aa6'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:27:18.570841 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6c204dfa-fdee-4689-a68d-b06a3b06e681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e8a1b34b723424e8552493a6c582aa6'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:27:18.572623 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:27:18.588914 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1370/2767] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:27:18 2022] DELETE /identity/v3/users/0e8a1b34b723424e8552493a6c582aa6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:27:18.773418 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2cd72-72b1-4323-b514-a80f2eb26161 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:27:18.774900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-58c2cd72-72b1-4323-b514-a80f2eb26161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:27:18.792207 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2cd72-72b1-4323-b514-a80f2eb26161 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:27:18.813480 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:27:18.852231 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec152bd-7734-4865-bee0-99248c41ddc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:27:18.853122 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec152bd-7734-4865-bee0-99248c41ddc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:27:18.853407 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec152bd-7734-4865-bee0-99248c41ddc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:27:18.853706 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec152bd-7734-4865-bee0-99248c41ddc5 admin admin] PATH_INFO: `/v3/projects/658a622dbd9a48fbb82208c50ab930d3` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:27:18.854984 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec152bd-7734-4865-bee0-99248c41ddc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=658a622dbd9a48fbb82208c50ab930d3)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:27:18.856931 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec152bd-7734-4865-bee0-99248c41ddc5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:27:18.910161 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3ec152bd-7734-4865-bee0-99248c41ddc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '658a622dbd9a48fbb82208c50ab930d3', 'request_id': 'req-3ec152bd-7734-4865-bee0-99248c41ddc5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:27:18.918043 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3ec152bd-7734-4865-bee0-99248c41ddc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '658a622dbd9a48fbb82208c50ab930d3', 'request_id': 'req-3ec152bd-7734-4865-bee0-99248c41ddc5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:27:18.924252 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3ec152bd-7734-4865-bee0-99248c41ddc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '658a622dbd9a48fbb82208c50ab930d3', 'request_id': 'req-3ec152bd-7734-4865-bee0-99248c41ddc5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:27:18.936101 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3ec152bd-7734-4865-bee0-99248c41ddc5 admin admin] The token cache is being invalidate because project 658a622dbd9a48fbb82208c50ab930d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:27:18.936406 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3ec152bd-7734-4865-bee0-99248c41ddc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:27:18.937093 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3ec152bd-7734-4865-bee0-99248c41ddc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:27:18.938990 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1398/2768] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:27:18 2022] DELETE /identity/v3/projects/658a622dbd9a48fbb82208c50ab930d3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:27:28.590118 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:27:28.934867 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:18.909387 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c204dfa-fdee-4689-a68d-b06a3b06e681 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:18.911454 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6c204dfa-fdee-4689-a68d-b06a3b06e681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:18.972275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec152bd-7734-4865-bee0-99248c41ddc5 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:18.975661 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3ec152bd-7734-4865-bee0-99248c41ddc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:19.002777 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c204dfa-fdee-4689-a68d-b06a3b06e681 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:19.052753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec152bd-7734-4865-bee0-99248c41ddc5 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:19.058980 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d053c2-2b51-456b-8d6e-ebcbd9a0ce1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:19.060098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d053c2-2b51-456b-8d6e-ebcbd9a0ce1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:19.060322 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d053c2-2b51-456b-8d6e-ebcbd9a0ce1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:19.060601 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d053c2-2b51-456b-8d6e-ebcbd9a0ce1f admin admin] PATH_INFO: `/v3/users/6eb164ecd85f498f9c3b3aabe90dc054` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:19.061831 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d053c2-2b51-456b-8d6e-ebcbd9a0ce1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eb164ecd85f498f9c3b3aabe90dc054)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:19.063899 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d053c2-2b51-456b-8d6e-ebcbd9a0ce1f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:19.073071 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51fc19-437a-4487-967b-1d8b71cbadbe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5fa9b0b0314ee9bb7e16b8464307b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:19.074139 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51fc19-437a-4487-967b-1d8b71cbadbe service placement] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:19.074593 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51fc19-437a-4487-967b-1d8b71cbadbe service placement] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:19.080303 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51fc19-437a-4487-967b-1d8b71cbadbe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:19.082550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7c51fc19-437a-4487-967b-1d8b71cbadbe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:19.096267 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51fc19-437a-4487-967b-1d8b71cbadbe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:19.097970 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51fc19-437a-4487-967b-1d8b71cbadbe service placement] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:19.109995 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1371/2769] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:28:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:28:19.124568 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-92d053c2-2b51-456b-8d6e-ebcbd9a0ce1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eb164ecd85f498f9c3b3aabe90dc054'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:19.124950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-92d053c2-2b51-456b-8d6e-ebcbd9a0ce1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eb164ecd85f498f9c3b3aabe90dc054'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:19.125283 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-92d053c2-2b51-456b-8d6e-ebcbd9a0ce1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eb164ecd85f498f9c3b3aabe90dc054'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:19.143235 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-92d053c2-2b51-456b-8d6e-ebcbd9a0ce1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eb164ecd85f498f9c3b3aabe90dc054'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:19.150998 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-92d053c2-2b51-456b-8d6e-ebcbd9a0ce1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eb164ecd85f498f9c3b3aabe90dc054'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:19.165078 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1399/2770] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:28:18 2022] DELETE /identity/v3/users/6eb164ecd85f498f9c3b3aabe90dc054 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:28:19.347254 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51fc19-437a-4487-967b-1d8b71cbadbe service placement] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:19.359130 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51fc19-437a-4487-967b-1d8b71cbadbe service placement] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:19.418183 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170b484-8b66-4a99-b529-76808b5ead0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:19.418988 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170b484-8b66-4a99-b529-76808b5ead0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:19.419271 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170b484-8b66-4a99-b529-76808b5ead0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:19.419542 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170b484-8b66-4a99-b529-76808b5ead0f admin admin] PATH_INFO: `/v3/projects/412630dcb5ff4402983a02b249583995` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:19.420727 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170b484-8b66-4a99-b529-76808b5ead0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=412630dcb5ff4402983a02b249583995)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:19.422636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170b484-8b66-4a99-b529-76808b5ead0f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:19.464701 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:19.476330 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5170b484-8b66-4a99-b529-76808b5ead0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '412630dcb5ff4402983a02b249583995', 'request_id': 'req-5170b484-8b66-4a99-b529-76808b5ead0f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:19.484636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5170b484-8b66-4a99-b529-76808b5ead0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '412630dcb5ff4402983a02b249583995', 'request_id': 'req-5170b484-8b66-4a99-b529-76808b5ead0f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:19.490266 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5170b484-8b66-4a99-b529-76808b5ead0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '412630dcb5ff4402983a02b249583995', 'request_id': 'req-5170b484-8b66-4a99-b529-76808b5ead0f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:19.501128 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5170b484-8b66-4a99-b529-76808b5ead0f admin admin] The token cache is being invalidate because project 412630dcb5ff4402983a02b249583995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:28:19.501412 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5170b484-8b66-4a99-b529-76808b5ead0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:19.502089 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5170b484-8b66-4a99-b529-76808b5ead0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:19.503619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1372/2771] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:28:19 2022] DELETE /identity/v3/projects/412630dcb5ff4402983a02b249583995 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:19.508984 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169d869-a799-4ca3-b7f1-8f1b6fc78fa3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:19.509275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169d869-a799-4ca3-b7f1-8f1b6fc78fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:19.509553 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169d869-a799-4ca3-b7f1-8f1b6fc78fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:19.527701 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8169d869-a799-4ca3-b7f1-8f1b6fc78fa3 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:19.528835 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8169d869-a799-4ca3-b7f1-8f1b6fc78fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:19.591933 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1400/2772] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:28:19.600176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce9d1d-6227-4822-831d-6dfa4e689f56 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:19.600468 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce9d1d-6227-4822-831d-6dfa4e689f56 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:19.600742 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce9d1d-6227-4822-831d-6dfa4e689f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:19.623286 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1fce9d1d-6227-4822-831d-6dfa4e689f56 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:19.624907 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1fce9d1d-6227-4822-831d-6dfa4e689f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:19.635520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1373/2773] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:28:19.641391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8169d869-a799-4ca3-b7f1-8f1b6fc78fa3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:19.651766 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8169d869-a799-4ca3-b7f1-8f1b6fc78fa3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:19.657491 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd12c8-31c6-413d-8756-96231e280421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:19.658182 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd12c8-31c6-413d-8756-96231e280421 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:19.658455 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd12c8-31c6-413d-8756-96231e280421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:19.658729 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd12c8-31c6-413d-8756-96231e280421 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:19.659462 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd12c8-31c6-413d-8756-96231e280421 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:28:19.659811 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd12c8-31c6-413d-8756-96231e280421 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:19.661419 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd12c8-31c6-413d-8756-96231e280421 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:19.673574 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1401/2774] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:28:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:28:19.677887 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce9d1d-6227-4822-831d-6dfa4e689f56 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:19.687400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce9d1d-6227-4822-831d-6dfa4e689f56 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:19.692996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd0af4-355c-4125-a6e2-e6672d581ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:19.693648 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd0af4-355c-4125-a6e2-e6672d581ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:19.693929 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd0af4-355c-4125-a6e2-e6672d581ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:19.694203 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd0af4-355c-4125-a6e2-e6672d581ff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:19.694891 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd0af4-355c-4125-a6e2-e6672d581ff7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:19.696841 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd0af4-355c-4125-a6e2-e6672d581ff7 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:19.725716 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1374/2775] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:28:19 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:28:19.730090 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd12c8-31c6-413d-8756-96231e280421 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:19.740709 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd12c8-31c6-413d-8756-96231e280421 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:19.744708 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:19.800286 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d9b6f-da45-4f7d-a639-e2a60dc82588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:19.800874 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d9b6f-da45-4f7d-a639-e2a60dc82588 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:19.801157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d9b6f-da45-4f7d-a639-e2a60dc82588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:19.801437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d9b6f-da45-4f7d-a639-e2a60dc82588 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:19.802127 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d9b6f-da45-4f7d-a639-e2a60dc82588 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:19.804110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d9b6f-da45-4f7d-a639-e2a60dc82588 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:19.826798 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1402/2776] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:28:19 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:28:19.831091 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd0af4-355c-4125-a6e2-e6672d581ff7 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:19.842061 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd0af4-355c-4125-a6e2-e6672d581ff7 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:19.847158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30d2e3-2383-4d39-8993-0c1b35b50dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:19.847903 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30d2e3-2383-4d39-8993-0c1b35b50dca admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:19.848163 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30d2e3-2383-4d39-8993-0c1b35b50dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:19.848434 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30d2e3-2383-4d39-8993-0c1b35b50dca admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:19.849157 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30d2e3-2383-4d39-8993-0c1b35b50dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:19.850750 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30d2e3-2383-4d39-8993-0c1b35b50dca admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:19.861281 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1375/2777] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:28:19.865417 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d9b6f-da45-4f7d-a639-e2a60dc82588 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:19.875900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d9b6f-da45-4f7d-a639-e2a60dc82588 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:19.881242 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6851c-8e57-4b6a-a446-6e555aee74a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:19.882038 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6851c-8e57-4b6a-a446-6e555aee74a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:19.882408 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6851c-8e57-4b6a-a446-6e555aee74a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:19.882688 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6851c-8e57-4b6a-a446-6e555aee74a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:19.883448 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6851c-8e57-4b6a-a446-6e555aee74a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:19.885235 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6851c-8e57-4b6a-a446-6e555aee74a0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:19.895704 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1403/2778] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:28:19.900075 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30d2e3-2383-4d39-8993-0c1b35b50dca admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:19.910583 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30d2e3-2383-4d39-8993-0c1b35b50dca admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:19.916413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b76a75-6555-4bce-8822-71e7614fd713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:19.917296 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b76a75-6555-4bce-8822-71e7614fd713 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:19.917602 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b76a75-6555-4bce-8822-71e7614fd713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:19.917876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b76a75-6555-4bce-8822-71e7614fd713 admin admin] PATH_INFO: `/v3/projects/de0b33b49efa4c049f177e03ce768686/users/582dbe6c9c874bfeb062d2a2a77516c9/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:19.933824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b76a75-6555-4bce-8822-71e7614fd713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de0b33b49efa4c049f177e03ce768686, user_id=582dbe6c9c874bfeb062d2a2a77516c9, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:19.936585 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b76a75-6555-4bce-8822-71e7614fd713 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:19.953144 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1376/2779] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:28:19 2022] PUT /identity/v3/projects/de0b33b49efa4c049f177e03ce768686/users/582dbe6c9c874bfeb062d2a2a77516c9/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:19.963298 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffbb6d-f757-4d3c-8b31-2dd1751c461e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:19.963593 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffbb6d-f757-4d3c-8b31-2dd1751c461e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:19.963898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffbb6d-f757-4d3c-8b31-2dd1751c461e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:19.992729 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-71ffbb6d-f757-4d3c-8b31-2dd1751c461e None None] MFA Rules not processed for user `582dbe6c9c874bfeb062d2a2a77516c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:19.993978 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-71ffbb6d-f757-4d3c-8b31-2dd1751c461e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:20.062960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1404/2780] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:28:20.074342 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b76a75-6555-4bce-8822-71e7614fd713 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:20.076333 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-62b76a75-6555-4bce-8822-71e7614fd713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:20.153896 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b76a75-6555-4bce-8822-71e7614fd713 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:20.160007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d0541-bd10-4b17-af08-941504de78ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:20.160651 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d0541-bd10-4b17-af08-941504de78ad service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:20.160940 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d0541-bd10-4b17-af08-941504de78ad service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:20.161200 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d0541-bd10-4b17-af08-941504de78ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:20.170168 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d0541-bd10-4b17-af08-941504de78ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:20.171826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d0541-bd10-4b17-af08-941504de78ad service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:20.181352 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1377/2781] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:28:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:28:27.496476 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffbb6d-f757-4d3c-8b31-2dd1751c461e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:27.579273 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffbb6d-f757-4d3c-8b31-2dd1751c461e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:27.584196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5770736-5852-4303-93f9-1a007c147e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:27.585082 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5770736-5852-4303-93f9-1a007c147e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:27.585361 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5770736-5852-4303-93f9-1a007c147e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:27.585637 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5770736-5852-4303-93f9-1a007c147e39 admin admin] PATH_INFO: `/v3/users/582dbe6c9c874bfeb062d2a2a77516c9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:27.586880 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5770736-5852-4303-93f9-1a007c147e39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=582dbe6c9c874bfeb062d2a2a77516c9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:27.588860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5770736-5852-4303-93f9-1a007c147e39 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:27.606454 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:27.641401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e5770736-5852-4303-93f9-1a007c147e39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '582dbe6c9c874bfeb062d2a2a77516c9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:27.642079 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e5770736-5852-4303-93f9-1a007c147e39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '582dbe6c9c874bfeb062d2a2a77516c9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:27.642726 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e5770736-5852-4303-93f9-1a007c147e39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '582dbe6c9c874bfeb062d2a2a77516c9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:27.656343 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e5770736-5852-4303-93f9-1a007c147e39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '582dbe6c9c874bfeb062d2a2a77516c9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:27.663901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e5770736-5852-4303-93f9-1a007c147e39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '582dbe6c9c874bfeb062d2a2a77516c9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:27.683244 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1405/2782] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:28:27 2022] DELETE /identity/v3/users/582dbe6c9c874bfeb062d2a2a77516c9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:27.689474 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd167281-5de3-44e6-9c8c-205512786a78 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:27.689983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd167281-5de3-44e6-9c8c-205512786a78 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:27.690097 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd167281-5de3-44e6-9c8c-205512786a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:27.707755 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-fd167281-5de3-44e6-9c8c-205512786a78 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:27.708969 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fd167281-5de3-44e6-9c8c-205512786a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:27.754686 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:27.778595 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1378/2783] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:28:27.788793 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5770736-5852-4303-93f9-1a007c147e39 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:27.790304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e5770736-5852-4303-93f9-1a007c147e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:27.848887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5770736-5852-4303-93f9-1a007c147e39 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:27.852793 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd843f9c-dedb-4d2b-926d-1607433a26d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:27.853450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd843f9c-dedb-4d2b-926d-1607433a26d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:27.853730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd843f9c-dedb-4d2b-926d-1607433a26d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:27.854007 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd843f9c-dedb-4d2b-926d-1607433a26d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:27.864186 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd843f9c-dedb-4d2b-926d-1607433a26d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:27.865655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd843f9c-dedb-4d2b-926d-1607433a26d7 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:27.875844 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1406/2784] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:28:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:28:28.170958 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd167281-5de3-44e6-9c8c-205512786a78 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:28.172715 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fd167281-5de3-44e6-9c8c-205512786a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:28.191979 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd167281-5de3-44e6-9c8c-205512786a78 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:28.198046 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee71df8-58f6-4621-8c0c-67a579a46507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:28.198922 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee71df8-58f6-4621-8c0c-67a579a46507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:28.199201 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee71df8-58f6-4621-8c0c-67a579a46507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:28.199487 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee71df8-58f6-4621-8c0c-67a579a46507 admin admin] PATH_INFO: `/v3/projects/de0b33b49efa4c049f177e03ce768686` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:28.200871 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee71df8-58f6-4621-8c0c-67a579a46507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de0b33b49efa4c049f177e03ce768686)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:28.202798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee71df8-58f6-4621-8c0c-67a579a46507 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:28.255323 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3ee71df8-58f6-4621-8c0c-67a579a46507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de0b33b49efa4c049f177e03ce768686', 'request_id': 'req-3ee71df8-58f6-4621-8c0c-67a579a46507'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:28.264066 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3ee71df8-58f6-4621-8c0c-67a579a46507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de0b33b49efa4c049f177e03ce768686', 'request_id': 'req-3ee71df8-58f6-4621-8c0c-67a579a46507'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:28.270076 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3ee71df8-58f6-4621-8c0c-67a579a46507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de0b33b49efa4c049f177e03ce768686', 'request_id': 'req-3ee71df8-58f6-4621-8c0c-67a579a46507'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:28.281284 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3ee71df8-58f6-4621-8c0c-67a579a46507 admin admin] The token cache is being invalidate because project de0b33b49efa4c049f177e03ce768686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:28:28.281561 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3ee71df8-58f6-4621-8c0c-67a579a46507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:28.282254 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3ee71df8-58f6-4621-8c0c-67a579a46507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:28.283867 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1379/2785] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:28:28 2022] DELETE /identity/v3/projects/de0b33b49efa4c049f177e03ce768686 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:28.288935 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d4783-5ebe-443e-bd75-0d23f2825d48 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:28.289243 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d4783-5ebe-443e-bd75-0d23f2825d48 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:28.289514 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d4783-5ebe-443e-bd75-0d23f2825d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:28.307336 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-000d4783-5ebe-443e-bd75-0d23f2825d48 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:28.308586 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-000d4783-5ebe-443e-bd75-0d23f2825d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:28.377719 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1407/2786] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:28:28.387114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2208b4-422b-4e82-8233-8179585ec62b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:28.387366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2208b4-422b-4e82-8233-8179585ec62b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:28.387729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2208b4-422b-4e82-8233-8179585ec62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:28.406345 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-bf2208b4-422b-4e82-8233-8179585ec62b None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:28.407547 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-bf2208b4-422b-4e82-8233-8179585ec62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:28.416025 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1380/2787] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:28:28.420329 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d4783-5ebe-443e-bd75-0d23f2825d48 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:28.430592 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d4783-5ebe-443e-bd75-0d23f2825d48 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:28.435910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b73c3e-21b9-4e63-a5ef-805085e2d77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:28.436577 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b73c3e-21b9-4e63-a5ef-805085e2d77f admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:28.436866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b73c3e-21b9-4e63-a5ef-805085e2d77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:28.437138 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b73c3e-21b9-4e63-a5ef-805085e2d77f admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:28.437874 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b73c3e-21b9-4e63-a5ef-805085e2d77f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:28:28.438180 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b73c3e-21b9-4e63-a5ef-805085e2d77f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:28.439831 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b73c3e-21b9-4e63-a5ef-805085e2d77f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:28.452216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1408/2788] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:28:28.456520 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2208b4-422b-4e82-8233-8179585ec62b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:28.466248 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2208b4-422b-4e82-8233-8179585ec62b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:28.472022 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e0984-2efc-4933-8ed7-e380180528c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:28.472679 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e0984-2efc-4933-8ed7-e380180528c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:28.472962 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e0984-2efc-4933-8ed7-e380180528c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:28.473235 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e0984-2efc-4933-8ed7-e380180528c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:28.473923 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e0984-2efc-4933-8ed7-e380180528c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:28.476008 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e0984-2efc-4933-8ed7-e380180528c1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:28.506236 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1381/2789] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:28:28 2022] POST /identity/v3/projects => generated 357 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:28:28.511973 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b73c3e-21b9-4e63-a5ef-805085e2d77f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:28.524619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b73c3e-21b9-4e63-a5ef-805085e2d77f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:28.553604 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:28.591789 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6c742-abaa-45ad-8a16-809a79f6c7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:28.592460 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6c742-abaa-45ad-8a16-809a79f6c7aa admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:28.592740 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6c742-abaa-45ad-8a16-809a79f6c7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:28.593013 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6c742-abaa-45ad-8a16-809a79f6c7aa admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:28.593673 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c6c742-abaa-45ad-8a16-809a79f6c7aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:28.595637 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c6c742-abaa-45ad-8a16-809a79f6c7aa admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:28.616863 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1409/2790] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:28:28 2022] POST /identity/v3/users => generated 279 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:28:28.621152 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e0984-2efc-4933-8ed7-e380180528c1 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:28.632357 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e0984-2efc-4933-8ed7-e380180528c1 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:28.637899 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd6cba-22b4-463d-860e-37acf38d19bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:28.638561 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd6cba-22b4-463d-860e-37acf38d19bc admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:28.638841 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd6cba-22b4-463d-860e-37acf38d19bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:28.639109 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd6cba-22b4-463d-860e-37acf38d19bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:28.639876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd6cba-22b4-463d-860e-37acf38d19bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:28.641517 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd6cba-22b4-463d-860e-37acf38d19bc admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:28.652417 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1382/2791] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:28:28.658230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6c742-abaa-45ad-8a16-809a79f6c7aa admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:28.673936 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6c742-abaa-45ad-8a16-809a79f6c7aa admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:28.681010 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261703a7-3db4-404e-b67b-9b2acc14a661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:28.681692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261703a7-3db4-404e-b67b-9b2acc14a661 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:28.681954 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261703a7-3db4-404e-b67b-9b2acc14a661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:28.682227 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261703a7-3db4-404e-b67b-9b2acc14a661 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:28.683153 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261703a7-3db4-404e-b67b-9b2acc14a661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:28.684601 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261703a7-3db4-404e-b67b-9b2acc14a661 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:28.696126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1410/2792] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:28:28.700444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd6cba-22b4-463d-860e-37acf38d19bc admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:28.711584 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd6cba-22b4-463d-860e-37acf38d19bc admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:28.716953 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9bc6c-c0ca-4147-9007-34238ba7855a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:28.717872 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9bc6c-c0ca-4147-9007-34238ba7855a admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:28.718151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9bc6c-c0ca-4147-9007-34238ba7855a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:28.718422 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9bc6c-c0ca-4147-9007-34238ba7855a admin admin] PATH_INFO: `/v3/projects/3bc9fd477de941118bc681ce410dc9c4/users/84c211b004404734820f5d5ab341b7d1/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:28.734253 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a9bc6c-c0ca-4147-9007-34238ba7855a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc9fd477de941118bc681ce410dc9c4, user_id=84c211b004404734820f5d5ab341b7d1, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:28.736988 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a9bc6c-c0ca-4147-9007-34238ba7855a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:28.752602 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1383/2793] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:28:28 2022] PUT /identity/v3/projects/3bc9fd477de941118bc681ce410dc9c4/users/84c211b004404734820f5d5ab341b7d1/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:28:28.763457 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466873e5-df71-4395-9541-674587efa2f4 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:28.763670 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466873e5-df71-4395-9541-674587efa2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:28.763977 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466873e5-df71-4395-9541-674587efa2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:28.790898 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-466873e5-df71-4395-9541-674587efa2f4 None None] MFA Rules not processed for user `84c211b004404734820f5d5ab341b7d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:28.792139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-466873e5-df71-4395-9541-674587efa2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:28.856924 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1411/2794] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:28 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:28:28.861467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9bc6c-c0ca-4147-9007-34238ba7855a admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:28.930441 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9bc6c-c0ca-4147-9007-34238ba7855a admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:28.939073 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f75097-0d67-4ac2-ac48-e48c2377f2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:28.939832 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f75097-0d67-4ac2-ac48-e48c2377f2b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:28.940118 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f75097-0d67-4ac2-ac48-e48c2377f2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:28.940391 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f75097-0d67-4ac2-ac48-e48c2377f2b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:28.941093 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f75097-0d67-4ac2-ac48-e48c2377f2b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:28.943194 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f75097-0d67-4ac2-ac48-e48c2377f2b1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:28.953830 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:28.971828 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1384/2795] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:28:28 2022] POST /identity/v3/projects => generated 355 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:28:28.976042 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466873e5-df71-4395-9541-674587efa2f4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:28.987375 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466873e5-df71-4395-9541-674587efa2f4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:29.048727 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b90c22-4862-4a94-84f3-c896dfeb2945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:29.049324 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b90c22-4862-4a94-84f3-c896dfeb2945 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:29.049602 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b90c22-4862-4a94-84f3-c896dfeb2945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:29.049878 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b90c22-4862-4a94-84f3-c896dfeb2945 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:29.050579 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b90c22-4862-4a94-84f3-c896dfeb2945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:29.052599 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b90c22-4862-4a94-84f3-c896dfeb2945 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:29.075337 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1412/2796] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:28:28 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:28:29.079427 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f75097-0d67-4ac2-ac48-e48c2377f2b1 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:29.090304 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f75097-0d67-4ac2-ac48-e48c2377f2b1 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:29.095196 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8122b2-6ef1-4085-b735-92aeb0b0468c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:29.095894 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8122b2-6ef1-4085-b735-92aeb0b0468c admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:29.096178 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8122b2-6ef1-4085-b735-92aeb0b0468c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:29.096449 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8122b2-6ef1-4085-b735-92aeb0b0468c admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:29.097176 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8122b2-6ef1-4085-b735-92aeb0b0468c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:29.098778 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8122b2-6ef1-4085-b735-92aeb0b0468c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:29.109515 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1385/2797] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:28:29.113585 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b90c22-4862-4a94-84f3-c896dfeb2945 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:29.124058 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b90c22-4862-4a94-84f3-c896dfeb2945 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:29.129941 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747872a-2001-4f2b-b3df-4e1ab118a8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:29.130861 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747872a-2001-4f2b-b3df-4e1ab118a8dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:29.131147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747872a-2001-4f2b-b3df-4e1ab118a8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:29.131408 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747872a-2001-4f2b-b3df-4e1ab118a8dd admin admin] PATH_INFO: `/v3/projects/50d878c6a33d443cb6ab825af37497de/users/3d7b69c301f34dfd80ee74e120ad711a/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:29.147791 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747872a-2001-4f2b-b3df-4e1ab118a8dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=50d878c6a33d443cb6ab825af37497de, user_id=3d7b69c301f34dfd80ee74e120ad711a, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:29.150538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747872a-2001-4f2b-b3df-4e1ab118a8dd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:29.160685 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:29.166689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1413/2798] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:28:29 2022] PUT /identity/v3/projects/50d878c6a33d443cb6ab825af37497de/users/3d7b69c301f34dfd80ee74e120ad711a/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:29.170711 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8122b2-6ef1-4085-b735-92aeb0b0468c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:29.181081 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8122b2-6ef1-4085-b735-92aeb0b0468c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:29.246364 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c166b62-62f0-4225-8921-73a5c2da2455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:29.247044 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c166b62-62f0-4225-8921-73a5c2da2455 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:29.247331 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c166b62-62f0-4225-8921-73a5c2da2455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:29.247635 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c166b62-62f0-4225-8921-73a5c2da2455 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:29.248370 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c166b62-62f0-4225-8921-73a5c2da2455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:29.249994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c166b62-62f0-4225-8921-73a5c2da2455 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:29.260490 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1386/2799] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:28:29.265160 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747872a-2001-4f2b-b3df-4e1ab118a8dd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:29.276223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747872a-2001-4f2b-b3df-4e1ab118a8dd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:29.281693 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8db7b-2dbb-4f04-83b8-187dd91007c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:29.282597 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8db7b-2dbb-4f04-83b8-187dd91007c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:29.282880 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8db7b-2dbb-4f04-83b8-187dd91007c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:29.283157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8db7b-2dbb-4f04-83b8-187dd91007c9 admin admin] PATH_INFO: `/v3/projects/50d878c6a33d443cb6ab825af37497de/users/3d7b69c301f34dfd80ee74e120ad711a/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:29.285400 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8db7b-2dbb-4f04-83b8-187dd91007c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50d878c6a33d443cb6ab825af37497de, user_id=3d7b69c301f34dfd80ee74e120ad711a, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:29.288122 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8db7b-2dbb-4f04-83b8-187dd91007c9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:29.298945 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1414/2800] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:28:29 2022] PUT /identity/v3/projects/50d878c6a33d443cb6ab825af37497de/users/3d7b69c301f34dfd80ee74e120ad711a/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:29.309248 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a547e7-8769-4ed6-94a4-927df11e1892 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:29.309559 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a547e7-8769-4ed6-94a4-927df11e1892 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:29.309834 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a547e7-8769-4ed6-94a4-927df11e1892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:29.338354 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b7a547e7-8769-4ed6-94a4-927df11e1892 None None] MFA Rules not processed for user `3d7b69c301f34dfd80ee74e120ad711a`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:29.339575 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b7a547e7-8769-4ed6-94a4-927df11e1892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:29.411204 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1387/2801] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:29 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:28:29.424404 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8db7b-2dbb-4f04-83b8-187dd91007c9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:29.426219 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-dfc8db7b-2dbb-4f04-83b8-187dd91007c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:29.484692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8db7b-2dbb-4f04-83b8-187dd91007c9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:29.488775 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f7629-c1e4-4e62-bb25-2c534d0cc832 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:29.489412 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f7629-c1e4-4e62-bb25-2c534d0cc832 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:29.489689 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f7629-c1e4-4e62-bb25-2c534d0cc832 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:29.489983 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f7629-c1e4-4e62-bb25-2c534d0cc832 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:29.499866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f7629-c1e4-4e62-bb25-2c534d0cc832 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:29.501315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f7629-c1e4-4e62-bb25-2c534d0cc832 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:29.511312 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1415/2802] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:28:29 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:28:29.586179 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a547e7-8769-4ed6-94a4-927df11e1892 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:29.596522 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a547e7-8769-4ed6-94a4-927df11e1892 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:29.600688 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0914d06d-c3eb-4d84-9424-e6b26f196c51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:29.601382 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914d06d-c3eb-4d84-9424-e6b26f196c51 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:29.601668 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914d06d-c3eb-4d84-9424-e6b26f196c51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:29.601945 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914d06d-c3eb-4d84-9424-e6b26f196c51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:29.662793 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0914d06d-c3eb-4d84-9424-e6b26f196c51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:29.664289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0914d06d-c3eb-4d84-9424-e6b26f196c51 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:29.668142 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:29.676482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1388/2803] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:28:29 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:28:39.507442 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=32,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:39.677280 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:44.590099 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f7629-c1e4-4e62-bb25-2c534d0cc832 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:44.704171 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f7629-c1e4-4e62-bb25-2c534d0cc832 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:44.709507 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4902eb-133a-45da-988a-0a5ea9c8be05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:44.710397 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4902eb-133a-45da-988a-0a5ea9c8be05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:44.710679 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4902eb-133a-45da-988a-0a5ea9c8be05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:44.710956 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4902eb-133a-45da-988a-0a5ea9c8be05 admin admin] PATH_INFO: `/v3/users/84c211b004404734820f5d5ab341b7d1` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:44.712164 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4902eb-133a-45da-988a-0a5ea9c8be05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84c211b004404734820f5d5ab341b7d1)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:44.714092 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4902eb-133a-45da-988a-0a5ea9c8be05 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:44.770567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-dd4902eb-133a-45da-988a-0a5ea9c8be05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84c211b004404734820f5d5ab341b7d1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:44.771213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-dd4902eb-133a-45da-988a-0a5ea9c8be05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84c211b004404734820f5d5ab341b7d1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:44.771874 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-dd4902eb-133a-45da-988a-0a5ea9c8be05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84c211b004404734820f5d5ab341b7d1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:44.783985 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-dd4902eb-133a-45da-988a-0a5ea9c8be05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84c211b004404734820f5d5ab341b7d1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:44.791349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-dd4902eb-133a-45da-988a-0a5ea9c8be05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84c211b004404734820f5d5ab341b7d1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:44.806279 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1416/2804] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:28:44 2022] DELETE /identity/v3/users/84c211b004404734820f5d5ab341b7d1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:44.810883 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d86f2-9733-4e23-a2f3-ace06bcb0286 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:44.811395 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d86f2-9733-4e23-a2f3-ace06bcb0286 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:44.811395 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d86f2-9733-4e23-a2f3-ace06bcb0286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:44.863038 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-475d86f2-9733-4e23-a2f3-ace06bcb0286 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:44.864200 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-475d86f2-9733-4e23-a2f3-ace06bcb0286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:44.925116 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1389/2805] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:28:44.935241 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4902eb-133a-45da-988a-0a5ea9c8be05 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:44.937094 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-dd4902eb-133a-45da-988a-0a5ea9c8be05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:44.983875 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:45.041492 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4902eb-133a-45da-988a-0a5ea9c8be05 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:45.046126 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005099c-895a-4902-89ef-f33ba332af9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:45.046791 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005099c-895a-4902-89ef-f33ba332af9b service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:45.047070 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005099c-895a-4902-89ef-f33ba332af9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:45.047344 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005099c-895a-4902-89ef-f33ba332af9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:45.057661 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005099c-895a-4902-89ef-f33ba332af9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:45.059163 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005099c-895a-4902-89ef-f33ba332af9b service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:45.069190 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1417/2806] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:28:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:28:45.347171 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d86f2-9733-4e23-a2f3-ace06bcb0286 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:45.348939 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-475d86f2-9733-4e23-a2f3-ace06bcb0286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:45.362300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d86f2-9733-4e23-a2f3-ace06bcb0286 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:45.368308 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae7e70-7d04-4b4f-9bda-973a048b405f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:45.369232 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae7e70-7d04-4b4f-9bda-973a048b405f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:45.369516 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae7e70-7d04-4b4f-9bda-973a048b405f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:45.369932 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae7e70-7d04-4b4f-9bda-973a048b405f admin admin] PATH_INFO: `/v3/projects/3bc9fd477de941118bc681ce410dc9c4` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:45.371288 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae7e70-7d04-4b4f-9bda-973a048b405f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bc9fd477de941118bc681ce410dc9c4)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:45.373322 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae7e70-7d04-4b4f-9bda-973a048b405f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:45.425853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eeae7e70-7d04-4b4f-9bda-973a048b405f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bc9fd477de941118bc681ce410dc9c4', 'request_id': 'req-eeae7e70-7d04-4b4f-9bda-973a048b405f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:45.433915 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eeae7e70-7d04-4b4f-9bda-973a048b405f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bc9fd477de941118bc681ce410dc9c4', 'request_id': 'req-eeae7e70-7d04-4b4f-9bda-973a048b405f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:45.439567 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eeae7e70-7d04-4b4f-9bda-973a048b405f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bc9fd477de941118bc681ce410dc9c4', 'request_id': 'req-eeae7e70-7d04-4b4f-9bda-973a048b405f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:45.450582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eeae7e70-7d04-4b4f-9bda-973a048b405f admin admin] The token cache is being invalidate because project 3bc9fd477de941118bc681ce410dc9c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:28:45.450906 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eeae7e70-7d04-4b4f-9bda-973a048b405f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:45.451518 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eeae7e70-7d04-4b4f-9bda-973a048b405f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:45.453110 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1390/2807] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:28:45 2022] DELETE /identity/v3/projects/3bc9fd477de941118bc681ce410dc9c4 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:45.457359 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005099c-895a-4902-89ef-f33ba332af9b service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:45.467646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005099c-895a-4902-89ef-f33ba332af9b service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:45.533147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c59448-9551-4407-ba5d-8a1c49265276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:45.534129 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c59448-9551-4407-ba5d-8a1c49265276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:45.534409 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c59448-9551-4407-ba5d-8a1c49265276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:45.534682 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c59448-9551-4407-ba5d-8a1c49265276 admin admin] PATH_INFO: `/v3/users/3d7b69c301f34dfd80ee74e120ad711a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:45.535952 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c59448-9551-4407-ba5d-8a1c49265276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d7b69c301f34dfd80ee74e120ad711a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:45.537907 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c59448-9551-4407-ba5d-8a1c49265276 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:45.595443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-38c59448-9551-4407-ba5d-8a1c49265276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d7b69c301f34dfd80ee74e120ad711a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:45.596157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-38c59448-9551-4407-ba5d-8a1c49265276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d7b69c301f34dfd80ee74e120ad711a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:45.597019 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-38c59448-9551-4407-ba5d-8a1c49265276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d7b69c301f34dfd80ee74e120ad711a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:45.610161 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-38c59448-9551-4407-ba5d-8a1c49265276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d7b69c301f34dfd80ee74e120ad711a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:45.617610 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-38c59448-9551-4407-ba5d-8a1c49265276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d7b69c301f34dfd80ee74e120ad711a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:45.631929 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1418/2808] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:28:45 2022] DELETE /identity/v3/users/3d7b69c301f34dfd80ee74e120ad711a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:45.941316 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae7e70-7d04-4b4f-9bda-973a048b405f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:45.942877 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-eeae7e70-7d04-4b4f-9bda-973a048b405f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:45.956651 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae7e70-7d04-4b4f-9bda-973a048b405f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:46.011736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:46.018870 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a129b6-22b0-4541-8d73-e4721b8e133e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:46.019918 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a129b6-22b0-4541-8d73-e4721b8e133e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:46.020272 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a129b6-22b0-4541-8d73-e4721b8e133e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:46.020794 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a129b6-22b0-4541-8d73-e4721b8e133e admin admin] PATH_INFO: `/v3/projects/50d878c6a33d443cb6ab825af37497de` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:46.022488 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a129b6-22b0-4541-8d73-e4721b8e133e admin admin] RBAC: Authorizing `identity:delete_project(project_id=50d878c6a33d443cb6ab825af37497de)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:46.024707 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a129b6-22b0-4541-8d73-e4721b8e133e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:46.077762 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-59a129b6-22b0-4541-8d73-e4721b8e133e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50d878c6a33d443cb6ab825af37497de', 'request_id': 'req-59a129b6-22b0-4541-8d73-e4721b8e133e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:46.087955 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-59a129b6-22b0-4541-8d73-e4721b8e133e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50d878c6a33d443cb6ab825af37497de', 'request_id': 'req-59a129b6-22b0-4541-8d73-e4721b8e133e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:46.094256 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-59a129b6-22b0-4541-8d73-e4721b8e133e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50d878c6a33d443cb6ab825af37497de', 'request_id': 'req-59a129b6-22b0-4541-8d73-e4721b8e133e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:46.106142 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-59a129b6-22b0-4541-8d73-e4721b8e133e admin admin] The token cache is being invalidate because project 50d878c6a33d443cb6ab825af37497de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:28:46.106431 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-59a129b6-22b0-4541-8d73-e4721b8e133e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:46.107123 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-59a129b6-22b0-4541-8d73-e4721b8e133e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:46.108799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1391/2809] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:28:45 2022] DELETE /identity/v3/projects/50d878c6a33d443cb6ab825af37497de => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:46.113815 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34714e9a-c122-41a7-9ea7-2591c321405c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:46.114154 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34714e9a-c122-41a7-9ea7-2591c321405c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:46.114424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34714e9a-c122-41a7-9ea7-2591c321405c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:46.152242 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-34714e9a-c122-41a7-9ea7-2591c321405c None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:46.153788 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-34714e9a-c122-41a7-9ea7-2591c321405c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:46.158315 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:46.230794 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1419/2810] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:28:46.238674 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491b3bf-2f32-4263-b687-52c261467cd6 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:46.238964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491b3bf-2f32-4263-b687-52c261467cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:46.239238 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491b3bf-2f32-4263-b687-52c261467cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:46.259383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a491b3bf-2f32-4263-b687-52c261467cd6 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:46.260557 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a491b3bf-2f32-4263-b687-52c261467cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:46.269300 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1392/2811] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:28:46.273511 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34714e9a-c122-41a7-9ea7-2591c321405c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:46.283860 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34714e9a-c122-41a7-9ea7-2591c321405c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:46.289901 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4835a12-9a9b-42bd-a063-d45eb14d6a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:46.290564 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4835a12-9a9b-42bd-a063-d45eb14d6a88 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:46.290842 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4835a12-9a9b-42bd-a063-d45eb14d6a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:46.291110 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4835a12-9a9b-42bd-a063-d45eb14d6a88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:46.291877 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4835a12-9a9b-42bd-a063-d45eb14d6a88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:28:46.292187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4835a12-9a9b-42bd-a063-d45eb14d6a88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:46.293831 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4835a12-9a9b-42bd-a063-d45eb14d6a88 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:46.306362 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1420/2812] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:28:46.310673 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491b3bf-2f32-4263-b687-52c261467cd6 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:46.320215 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491b3bf-2f32-4263-b687-52c261467cd6 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:46.325382 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df94e8-c825-43f3-b9bc-097b7997b822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:46.326029 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df94e8-c825-43f3-b9bc-097b7997b822 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:46.326314 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df94e8-c825-43f3-b9bc-097b7997b822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:46.326586 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df94e8-c825-43f3-b9bc-097b7997b822 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:46.327292 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df94e8-c825-43f3-b9bc-097b7997b822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:46.329238 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df94e8-c825-43f3-b9bc-097b7997b822 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:46.358139 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1393/2813] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:28:46 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:28:46.362291 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4835a12-9a9b-42bd-a063-d45eb14d6a88 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:46.372897 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4835a12-9a9b-42bd-a063-d45eb14d6a88 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:46.432751 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03acfd0-121c-483c-a191-09f883627a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:46.433423 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03acfd0-121c-483c-a191-09f883627a38 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:46.433704 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03acfd0-121c-483c-a191-09f883627a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:46.433974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03acfd0-121c-483c-a191-09f883627a38 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:46.434653 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03acfd0-121c-483c-a191-09f883627a38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:46.436619 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03acfd0-121c-483c-a191-09f883627a38 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:46.462736 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1421/2814] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:28:46 2022] POST /identity/v3/users => generated 282 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:28:46.467140 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df94e8-c825-43f3-b9bc-097b7997b822 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:46.477952 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df94e8-c825-43f3-b9bc-097b7997b822 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:46.483321 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29b836-423e-4db0-b791-942d0bec3bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:46.484067 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29b836-423e-4db0-b791-942d0bec3bac admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:46.484348 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29b836-423e-4db0-b791-942d0bec3bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:46.484621 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29b836-423e-4db0-b791-942d0bec3bac admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:46.485354 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29b836-423e-4db0-b791-942d0bec3bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:46.487003 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29b836-423e-4db0-b791-942d0bec3bac admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:46.497833 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1394/2815] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:28:46.501808 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03acfd0-121c-483c-a191-09f883627a38 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:46.514204 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03acfd0-121c-483c-a191-09f883627a38 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:46.520680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1c4fa-fd9b-4251-a423-2cccc48cbf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:46.521694 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1c4fa-fd9b-4251-a423-2cccc48cbf13 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:46.522077 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1c4fa-fd9b-4251-a423-2cccc48cbf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:46.522453 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1c4fa-fd9b-4251-a423-2cccc48cbf13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:46.523560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1c4fa-fd9b-4251-a423-2cccc48cbf13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:46.526112 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1c4fa-fd9b-4251-a423-2cccc48cbf13 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:46.539321 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1422/2816] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:28:46.543636 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29b836-423e-4db0-b791-942d0bec3bac admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:46.554276 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29b836-423e-4db0-b791-942d0bec3bac admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:46.560022 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566151d-ba6d-4e91-8e91-f052889396ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:46.560909 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566151d-ba6d-4e91-8e91-f052889396ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:46.561191 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566151d-ba6d-4e91-8e91-f052889396ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:46.561467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566151d-ba6d-4e91-8e91-f052889396ff admin admin] PATH_INFO: `/v3/projects/945d3ad9912e4e3681d365a44d0cb725/users/ff4d653a8a6947c9a3278c1ae814cb57/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:46.576895 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566151d-ba6d-4e91-8e91-f052889396ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=945d3ad9912e4e3681d365a44d0cb725, user_id=ff4d653a8a6947c9a3278c1ae814cb57, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:46.579573 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566151d-ba6d-4e91-8e91-f052889396ff admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:46.595000 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1395/2817] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:28:46 2022] PUT /identity/v3/projects/945d3ad9912e4e3681d365a44d0cb725/users/ff4d653a8a6947c9a3278c1ae814cb57/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:46.605968 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d78f4-f333-472c-b5dc-9f9a25f42ed6 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:46.606478 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d78f4-f333-472c-b5dc-9f9a25f42ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:46.606576 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d78f4-f333-472c-b5dc-9f9a25f42ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:46.635124 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-af2d78f4-f333-472c-b5dc-9f9a25f42ed6 None None] MFA Rules not processed for user `ff4d653a8a6947c9a3278c1ae814cb57`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:46.636396 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-af2d78f4-f333-472c-b5dc-9f9a25f42ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:46.703998 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1423/2818] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:46 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:28:46.708678 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566151d-ba6d-4e91-8e91-f052889396ff admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:46.783150 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566151d-ba6d-4e91-8e91-f052889396ff admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:46.788393 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687d468-4f5b-46cd-84b3-815e8545e6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:46.789053 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687d468-4f5b-46cd-84b3-815e8545e6cb admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:46.789337 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687d468-4f5b-46cd-84b3-815e8545e6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:46.789609 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687d468-4f5b-46cd-84b3-815e8545e6cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:46.790305 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687d468-4f5b-46cd-84b3-815e8545e6cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:46.792268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687d468-4f5b-46cd-84b3-815e8545e6cb admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:46.819785 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1396/2819] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:28:46 2022] POST /identity/v3/projects => generated 365 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:28:46.823931 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d78f4-f333-472c-b5dc-9f9a25f42ed6 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:46.830681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:46.835583 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d78f4-f333-472c-b5dc-9f9a25f42ed6 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:46.901437 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5eb62b-c216-4ea3-9631-1fa26253aaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:46.902121 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5eb62b-c216-4ea3-9631-1fa26253aaef admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:46.902401 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5eb62b-c216-4ea3-9631-1fa26253aaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:46.902677 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5eb62b-c216-4ea3-9631-1fa26253aaef admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:46.903376 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5eb62b-c216-4ea3-9631-1fa26253aaef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:46.905411 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5eb62b-c216-4ea3-9631-1fa26253aaef admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:46.926090 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1424/2820] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:28:46 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:28:46.930153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687d468-4f5b-46cd-84b3-815e8545e6cb admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:46.941025 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687d468-4f5b-46cd-84b3-815e8545e6cb admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:46.946292 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916031d-75f4-4667-a362-9cf9915014c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:46.946915 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916031d-75f4-4667-a362-9cf9915014c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:46.947229 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916031d-75f4-4667-a362-9cf9915014c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:46.947507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916031d-75f4-4667-a362-9cf9915014c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:46.948273 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916031d-75f4-4667-a362-9cf9915014c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:46.949900 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916031d-75f4-4667-a362-9cf9915014c7 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:46.952065 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:46.967555 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1397/2821] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:28:46.968743 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5eb62b-c216-4ea3-9631-1fa26253aaef admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:46.979541 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5eb62b-c216-4ea3-9631-1fa26253aaef admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:46.985538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37ff7e-5ffe-449b-9c29-a52d0762439a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:46.986474 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37ff7e-5ffe-449b-9c29-a52d0762439a admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:46.986756 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37ff7e-5ffe-449b-9c29-a52d0762439a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:46.987071 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37ff7e-5ffe-449b-9c29-a52d0762439a admin admin] PATH_INFO: `/v3/projects/55bdaa08768e477dac6947cf766c0376/users/a7d20947696b41b6aef36d44d4351ff7/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:47.003742 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba37ff7e-5ffe-449b-9c29-a52d0762439a admin admin] RBAC: Authorizing `identity:create_grant(project_id=55bdaa08768e477dac6947cf766c0376, user_id=a7d20947696b41b6aef36d44d4351ff7, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:47.006816 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba37ff7e-5ffe-449b-9c29-a52d0762439a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:47.023095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1425/2822] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:28:46 2022] PUT /identity/v3/projects/55bdaa08768e477dac6947cf766c0376/users/a7d20947696b41b6aef36d44d4351ff7/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:28:47.027312 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916031d-75f4-4667-a362-9cf9915014c7 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:47.037718 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916031d-75f4-4667-a362-9cf9915014c7 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:47.097032 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fece61f-e237-440f-9d84-5f0f00c73960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:47.097709 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fece61f-e237-440f-9d84-5f0f00c73960 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:47.097983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fece61f-e237-440f-9d84-5f0f00c73960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:47.098299 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fece61f-e237-440f-9d84-5f0f00c73960 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:47.099028 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fece61f-e237-440f-9d84-5f0f00c73960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:47.100696 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fece61f-e237-440f-9d84-5f0f00c73960 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:47.111014 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1398/2823] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:28:47.115452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37ff7e-5ffe-449b-9c29-a52d0762439a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:47.126548 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37ff7e-5ffe-449b-9c29-a52d0762439a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:47.132301 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d0908-1062-41ae-b096-cd86d2cdd6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:47.133195 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d0908-1062-41ae-b096-cd86d2cdd6c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:47.133481 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d0908-1062-41ae-b096-cd86d2cdd6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:47.133756 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d0908-1062-41ae-b096-cd86d2cdd6c6 admin admin] PATH_INFO: `/v3/projects/55bdaa08768e477dac6947cf766c0376/users/a7d20947696b41b6aef36d44d4351ff7/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:47.135826 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d0908-1062-41ae-b096-cd86d2cdd6c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55bdaa08768e477dac6947cf766c0376, user_id=a7d20947696b41b6aef36d44d4351ff7, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:47.138489 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d0908-1062-41ae-b096-cd86d2cdd6c6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:47.149157 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1426/2824] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:28:47 2022] PUT /identity/v3/projects/55bdaa08768e477dac6947cf766c0376/users/a7d20947696b41b6aef36d44d4351ff7/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:47.160374 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5e2ee-9b3d-4c17-bb58-6f14e5ee94a1 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:47.160666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5e2ee-9b3d-4c17-bb58-6f14e5ee94a1 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:47.160943 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5e2ee-9b3d-4c17-bb58-6f14e5ee94a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:47.187589 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-41e5e2ee-9b3d-4c17-bb58-6f14e5ee94a1 None None] MFA Rules not processed for user `a7d20947696b41b6aef36d44d4351ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:47.188839 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-41e5e2ee-9b3d-4c17-bb58-6f14e5ee94a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:47.276133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1399/2825] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:47 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:28:47.290731 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d0908-1062-41ae-b096-cd86d2cdd6c6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:47.293387 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a87d0908-1062-41ae-b096-cd86d2cdd6c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:47.364967 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d0908-1062-41ae-b096-cd86d2cdd6c6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:47.369717 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079790d7-f7f0-46ec-9b61-f8488c90b6c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:47.370360 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079790d7-f7f0-46ec-9b61-f8488c90b6c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:47.370641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079790d7-f7f0-46ec-9b61-f8488c90b6c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:47.370911 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079790d7-f7f0-46ec-9b61-f8488c90b6c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:47.381226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079790d7-f7f0-46ec-9b61-f8488c90b6c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:47.382690 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079790d7-f7f0-46ec-9b61-f8488c90b6c2 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:47.392095 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1427/2826] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:28:47 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:28:47.472503 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5e2ee-9b3d-4c17-bb58-6f14e5ee94a1 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:47.483447 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5e2ee-9b3d-4c17-bb58-6f14e5ee94a1 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:47.488651 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616d953-f257-49e7-ac52-a6123fce3b21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:47.489295 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616d953-f257-49e7-ac52-a6123fce3b21 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:47.489582 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616d953-f257-49e7-ac52-a6123fce3b21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:47.489857 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616d953-f257-49e7-ac52-a6123fce3b21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:47.554861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616d953-f257-49e7-ac52-a6123fce3b21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:47.556289 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616d953-f257-49e7-ac52-a6123fce3b21 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:47.567206 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1400/2827] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:28:47 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:28:50.319324 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079790d7-f7f0-46ec-9b61-f8488c90b6c2 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:50.385024 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079790d7-f7f0-46ec-9b61-f8488c90b6c2 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:50.390144 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf12d34-95de-423f-b1b7-a92b96beb2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:50.391037 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf12d34-95de-423f-b1b7-a92b96beb2a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:50.391317 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf12d34-95de-423f-b1b7-a92b96beb2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:50.391585 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf12d34-95de-423f-b1b7-a92b96beb2a4 admin admin] PATH_INFO: `/v3/users/ff4d653a8a6947c9a3278c1ae814cb57` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:50.392838 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf12d34-95de-423f-b1b7-a92b96beb2a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff4d653a8a6947c9a3278c1ae814cb57)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:50.394828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf12d34-95de-423f-b1b7-a92b96beb2a4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:50.396340 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:50.444294 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6bf12d34-95de-423f-b1b7-a92b96beb2a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff4d653a8a6947c9a3278c1ae814cb57'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:50.445027 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6bf12d34-95de-423f-b1b7-a92b96beb2a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff4d653a8a6947c9a3278c1ae814cb57'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:50.445629 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6bf12d34-95de-423f-b1b7-a92b96beb2a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff4d653a8a6947c9a3278c1ae814cb57'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:50.457488 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6bf12d34-95de-423f-b1b7-a92b96beb2a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff4d653a8a6947c9a3278c1ae814cb57'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:50.464286 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6bf12d34-95de-423f-b1b7-a92b96beb2a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff4d653a8a6947c9a3278c1ae814cb57'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:50.478304 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1428/2828] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:28:50 2022] DELETE /identity/v3/users/ff4d653a8a6947c9a3278c1ae814cb57 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:50.482961 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cab225-f917-404c-9612-ea056fdb561b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:50.483438 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cab225-f917-404c-9612-ea056fdb561b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:50.483559 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cab225-f917-404c-9612-ea056fdb561b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:50.501441 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-49cab225-f917-404c-9612-ea056fdb561b None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:50.502676 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-49cab225-f917-404c-9612-ea056fdb561b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:50.516368 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:50.567568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1401/2829] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:28:50.582213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf12d34-95de-423f-b1b7-a92b96beb2a4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:50.584713 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6bf12d34-95de-423f-b1b7-a92b96beb2a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:50.649782 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf12d34-95de-423f-b1b7-a92b96beb2a4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:50.654216 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42e18c-fd0c-4f0b-a7e7-1e7f65d0f484 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:50.654873 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42e18c-fd0c-4f0b-a7e7-1e7f65d0f484 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:50.655156 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42e18c-fd0c-4f0b-a7e7-1e7f65d0f484 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:50.655431 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42e18c-fd0c-4f0b-a7e7-1e7f65d0f484 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:50.667147 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42e18c-fd0c-4f0b-a7e7-1e7f65d0f484 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:50.669558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42e18c-fd0c-4f0b-a7e7-1e7f65d0f484 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:50.682093 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1429/2830] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:28:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:28:50.960599 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cab225-f917-404c-9612-ea056fdb561b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:50.962123 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-49cab225-f917-404c-9612-ea056fdb561b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:50.974791 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cab225-f917-404c-9612-ea056fdb561b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:50.982217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988675f-7657-469c-b68e-7bbfd4daf66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:50.983245 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988675f-7657-469c-b68e-7bbfd4daf66c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:50.983527 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988675f-7657-469c-b68e-7bbfd4daf66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:50.983811 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988675f-7657-469c-b68e-7bbfd4daf66c admin admin] PATH_INFO: `/v3/projects/945d3ad9912e4e3681d365a44d0cb725` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:50.985092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988675f-7657-469c-b68e-7bbfd4daf66c admin admin] RBAC: Authorizing `identity:delete_project(project_id=945d3ad9912e4e3681d365a44d0cb725)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:50.987160 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988675f-7657-469c-b68e-7bbfd4daf66c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:51.039208 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2988675f-7657-469c-b68e-7bbfd4daf66c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '945d3ad9912e4e3681d365a44d0cb725', 'request_id': 'req-2988675f-7657-469c-b68e-7bbfd4daf66c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:51.047474 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2988675f-7657-469c-b68e-7bbfd4daf66c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '945d3ad9912e4e3681d365a44d0cb725', 'request_id': 'req-2988675f-7657-469c-b68e-7bbfd4daf66c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:51.053092 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2988675f-7657-469c-b68e-7bbfd4daf66c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '945d3ad9912e4e3681d365a44d0cb725', 'request_id': 'req-2988675f-7657-469c-b68e-7bbfd4daf66c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:51.064684 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2988675f-7657-469c-b68e-7bbfd4daf66c admin admin] The token cache is being invalidate because project 945d3ad9912e4e3681d365a44d0cb725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:28:51.064975 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2988675f-7657-469c-b68e-7bbfd4daf66c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:51.065608 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2988675f-7657-469c-b68e-7bbfd4daf66c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:51.067303 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1402/2831] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:28:50 2022] DELETE /identity/v3/projects/945d3ad9912e4e3681d365a44d0cb725 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:28:51.071558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42e18c-fd0c-4f0b-a7e7-1e7f65d0f484 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:51.081562 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42e18c-fd0c-4f0b-a7e7-1e7f65d0f484 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:51.140010 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc7395-d6a9-40f7-8563-c3651ffbb279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:51.140892 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc7395-d6a9-40f7-8563-c3651ffbb279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:51.141171 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc7395-d6a9-40f7-8563-c3651ffbb279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:51.141442 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc7395-d6a9-40f7-8563-c3651ffbb279 admin admin] PATH_INFO: `/v3/users/a7d20947696b41b6aef36d44d4351ff7` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:51.142612 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc7395-d6a9-40f7-8563-c3651ffbb279 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7d20947696b41b6aef36d44d4351ff7)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:51.144641 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc7395-d6a9-40f7-8563-c3651ffbb279 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:51.167572 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:51.195090 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d0cc7395-d6a9-40f7-8563-c3651ffbb279 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7d20947696b41b6aef36d44d4351ff7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:51.195779 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d0cc7395-d6a9-40f7-8563-c3651ffbb279 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7d20947696b41b6aef36d44d4351ff7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:51.196394 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d0cc7395-d6a9-40f7-8563-c3651ffbb279 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7d20947696b41b6aef36d44d4351ff7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:51.208441 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d0cc7395-d6a9-40f7-8563-c3651ffbb279 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7d20947696b41b6aef36d44d4351ff7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:51.215998 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d0cc7395-d6a9-40f7-8563-c3651ffbb279 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7d20947696b41b6aef36d44d4351ff7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:51.230614 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1430/2832] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:28:51 2022] DELETE /identity/v3/users/a7d20947696b41b6aef36d44d4351ff7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:51.510752 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988675f-7657-469c-b68e-7bbfd4daf66c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:51.512399 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2988675f-7657-469c-b68e-7bbfd4daf66c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:51.526365 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988675f-7657-469c-b68e-7bbfd4daf66c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:51.583071 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d857f-2d46-4efa-8421-35161c757ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:51.583911 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d857f-2d46-4efa-8421-35161c757ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:51.584193 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d857f-2d46-4efa-8421-35161c757ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:51.584467 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d857f-2d46-4efa-8421-35161c757ecb admin admin] PATH_INFO: `/v3/projects/55bdaa08768e477dac6947cf766c0376` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:51.585678 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d857f-2d46-4efa-8421-35161c757ecb admin admin] RBAC: Authorizing `identity:delete_project(project_id=55bdaa08768e477dac6947cf766c0376)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:51.587620 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d857f-2d46-4efa-8421-35161c757ecb admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:51.645232 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:51.650463 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-398d857f-2d46-4efa-8421-35161c757ecb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55bdaa08768e477dac6947cf766c0376', 'request_id': 'req-398d857f-2d46-4efa-8421-35161c757ecb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:51.660620 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-398d857f-2d46-4efa-8421-35161c757ecb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55bdaa08768e477dac6947cf766c0376', 'request_id': 'req-398d857f-2d46-4efa-8421-35161c757ecb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:51.666348 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-398d857f-2d46-4efa-8421-35161c757ecb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55bdaa08768e477dac6947cf766c0376', 'request_id': 'req-398d857f-2d46-4efa-8421-35161c757ecb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:51.677764 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-398d857f-2d46-4efa-8421-35161c757ecb admin admin] The token cache is being invalidate because project 55bdaa08768e477dac6947cf766c0376 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:28:51.678056 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-398d857f-2d46-4efa-8421-35161c757ecb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:51.678706 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-398d857f-2d46-4efa-8421-35161c757ecb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:51.680263 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1403/2833] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:28:51 2022] DELETE /identity/v3/projects/55bdaa08768e477dac6947cf766c0376 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:51.685354 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae900c-83c0-43e4-898e-c6bea0805c5a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:51.685880 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae900c-83c0-43e4-898e-c6bea0805c5a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:51.685880 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae900c-83c0-43e4-898e-c6bea0805c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:51.705019 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3cae900c-83c0-43e4-898e-c6bea0805c5a None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:51.706233 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3cae900c-83c0-43e4-898e-c6bea0805c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:51.768392 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1431/2834] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:28:51.776264 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac22993-7357-4c3e-9749-84df41c0870b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:51.776601 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac22993-7357-4c3e-9749-84df41c0870b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:51.776876 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac22993-7357-4c3e-9749-84df41c0870b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:51.796393 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5ac22993-7357-4c3e-9749-84df41c0870b None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:51.797609 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5ac22993-7357-4c3e-9749-84df41c0870b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:51.807624 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1404/2835] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:28:51.811912 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae900c-83c0-43e4-898e-c6bea0805c5a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:51.822388 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae900c-83c0-43e4-898e-c6bea0805c5a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:51.827584 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e9b2b-9211-4db3-a72b-0cb9990226a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:51.828275 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e9b2b-9211-4db3-a72b-0cb9990226a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:51.828558 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e9b2b-9211-4db3-a72b-0cb9990226a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:51.828837 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e9b2b-9211-4db3-a72b-0cb9990226a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:51.829568 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e9b2b-9211-4db3-a72b-0cb9990226a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:28:51.829879 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e9b2b-9211-4db3-a72b-0cb9990226a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:51.831492 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e9b2b-9211-4db3-a72b-0cb9990226a7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:51.843509 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1432/2836] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:28:51.847764 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac22993-7357-4c3e-9749-84df41c0870b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:51.857801 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac22993-7357-4c3e-9749-84df41c0870b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:51.863827 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5dc364-5de9-4de6-aa9e-d32f54df1661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:51.864506 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dc364-5de9-4de6-aa9e-d32f54df1661 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:51.864809 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dc364-5de9-4de6-aa9e-d32f54df1661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:51.865077 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dc364-5de9-4de6-aa9e-d32f54df1661 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:51.865775 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5dc364-5de9-4de6-aa9e-d32f54df1661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:51.867730 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5dc364-5de9-4de6-aa9e-d32f54df1661 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:51.895950 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1405/2837] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:28:51 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:28:51.900139 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e9b2b-9211-4db3-a72b-0cb9990226a7 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:51.910552 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e9b2b-9211-4db3-a72b-0cb9990226a7 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:51.975346 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7f299-84f7-4426-8528-b962c473423f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:51.975346 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7f299-84f7-4426-8528-b962c473423f admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:51.975346 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7f299-84f7-4426-8528-b962c473423f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:51.975346 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7f299-84f7-4426-8528-b962c473423f admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:51.975929 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7f299-84f7-4426-8528-b962c473423f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:51.977325 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7f299-84f7-4426-8528-b962c473423f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:51.999829 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1433/2838] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:28:51 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:28:52.004288 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5dc364-5de9-4de6-aa9e-d32f54df1661 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:52.015150 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5dc364-5de9-4de6-aa9e-d32f54df1661 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:52.020211 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8928bb62-e145-421d-ac1f-ada40dd44850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:52.020898 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928bb62-e145-421d-ac1f-ada40dd44850 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:52.021284 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928bb62-e145-421d-ac1f-ada40dd44850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:52.021565 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928bb62-e145-421d-ac1f-ada40dd44850 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:52.022315 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8928bb62-e145-421d-ac1f-ada40dd44850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:52.023966 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8928bb62-e145-421d-ac1f-ada40dd44850 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:52.034445 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1406/2839] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:28:52.039206 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7f299-84f7-4426-8528-b962c473423f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:52.053210 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7f299-84f7-4426-8528-b962c473423f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:52.058680 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e638a-8e7f-40a4-b592-a7a52c3c6105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:52.059344 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e638a-8e7f-40a4-b592-a7a52c3c6105 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:52.059645 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e638a-8e7f-40a4-b592-a7a52c3c6105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:52.059933 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e638a-8e7f-40a4-b592-a7a52c3c6105 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:52.060655 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e638a-8e7f-40a4-b592-a7a52c3c6105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:52.062271 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e638a-8e7f-40a4-b592-a7a52c3c6105 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:52.072370 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1434/2840] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:28:52.076607 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8928bb62-e145-421d-ac1f-ada40dd44850 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:52.087371 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8928bb62-e145-421d-ac1f-ada40dd44850 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:52.093240 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a058f-0054-4bce-8cf9-cb7e73065fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:52.094194 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a058f-0054-4bce-8cf9-cb7e73065fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:52.094366 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a058f-0054-4bce-8cf9-cb7e73065fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:52.094666 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a058f-0054-4bce-8cf9-cb7e73065fb4 admin admin] PATH_INFO: `/v3/projects/1cda401b9e5e47148522c769b056b47d/users/cae3abcec46c4ebbabba41770b705151/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:52.111635 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a058f-0054-4bce-8cf9-cb7e73065fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cda401b9e5e47148522c769b056b47d, user_id=cae3abcec46c4ebbabba41770b705151, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:52.114435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a058f-0054-4bce-8cf9-cb7e73065fb4 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:52.131057 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1407/2841] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:28:52 2022] PUT /identity/v3/projects/1cda401b9e5e47148522c769b056b47d/users/cae3abcec46c4ebbabba41770b705151/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:28:52.142725 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb658f-bcdc-4e7e-b272-c78f6f45bd52 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:52.143016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb658f-bcdc-4e7e-b272-c78f6f45bd52 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:52.143295 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb658f-bcdc-4e7e-b272-c78f6f45bd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:52.156057 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:52.171529 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-cfcb658f-bcdc-4e7e-b272-c78f6f45bd52 None None] MFA Rules not processed for user `cae3abcec46c4ebbabba41770b705151`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:52.172815 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cfcb658f-bcdc-4e7e-b272-c78f6f45bd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:52.236477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1435/2842] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:52 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:28:52.241305 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a058f-0054-4bce-8cf9-cb7e73065fb4 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:52.311824 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a058f-0054-4bce-8cf9-cb7e73065fb4 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:52.317857 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7aeec-2bdf-42a7-974a-9f3828cbc2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:52.318523 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7aeec-2bdf-42a7-974a-9f3828cbc2b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:52.318804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7aeec-2bdf-42a7-974a-9f3828cbc2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:52.319078 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7aeec-2bdf-42a7-974a-9f3828cbc2b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:52.319804 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d7aeec-2bdf-42a7-974a-9f3828cbc2b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:52.321738 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d7aeec-2bdf-42a7-974a-9f3828cbc2b0 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:52.350836 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1408/2843] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:28:52 2022] POST /identity/v3/projects => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:28:52.355009 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb658f-bcdc-4e7e-b272-c78f6f45bd52 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:52.366016 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb658f-bcdc-4e7e-b272-c78f6f45bd52 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:52.432696 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8ddc6-954e-4aee-9fbc-4cd0774433bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:52.433624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8ddc6-954e-4aee-9fbc-4cd0774433bd admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:52.434033 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8ddc6-954e-4aee-9fbc-4cd0774433bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:52.434416 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8ddc6-954e-4aee-9fbc-4cd0774433bd admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:52.435417 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8ddc6-954e-4aee-9fbc-4cd0774433bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:52.438412 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8ddc6-954e-4aee-9fbc-4cd0774433bd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:52.459809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1436/2844] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:28:52 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:28:52.463891 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7aeec-2bdf-42a7-974a-9f3828cbc2b0 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:52.474843 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7aeec-2bdf-42a7-974a-9f3828cbc2b0 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:52.479852 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be077d64-84b0-4bfc-ac59-50aa9a6ffffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:52.480513 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be077d64-84b0-4bfc-ac59-50aa9a6ffffe admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:52.480796 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be077d64-84b0-4bfc-ac59-50aa9a6ffffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:52.481071 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be077d64-84b0-4bfc-ac59-50aa9a6ffffe admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:52.481798 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be077d64-84b0-4bfc-ac59-50aa9a6ffffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:52.483413 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be077d64-84b0-4bfc-ac59-50aa9a6ffffe admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:52.494464 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1409/2845] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:28:52.498406 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8ddc6-954e-4aee-9fbc-4cd0774433bd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:52.509259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8ddc6-954e-4aee-9fbc-4cd0774433bd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:52.515104 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436d497-126b-4fe2-a5bb-b10d66b25f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:52.516033 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436d497-126b-4fe2-a5bb-b10d66b25f40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:52.516310 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436d497-126b-4fe2-a5bb-b10d66b25f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:52.516606 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436d497-126b-4fe2-a5bb-b10d66b25f40 admin admin] PATH_INFO: `/v3/projects/bd500b202e49473eb1b0c1dd788b3cd4/users/9c4a276bf8644541baa6d42b8a02bb2a/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:52.532029 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436d497-126b-4fe2-a5bb-b10d66b25f40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd500b202e49473eb1b0c1dd788b3cd4, user_id=9c4a276bf8644541baa6d42b8a02bb2a, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:52.534726 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436d497-126b-4fe2-a5bb-b10d66b25f40 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:52.549674 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1437/2846] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:28:52 2022] PUT /identity/v3/projects/bd500b202e49473eb1b0c1dd788b3cd4/users/9c4a276bf8644541baa6d42b8a02bb2a/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:28:52.553832 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be077d64-84b0-4bfc-ac59-50aa9a6ffffe admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:52.564435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be077d64-84b0-4bfc-ac59-50aa9a6ffffe admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:52.604638 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:52.626052 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48161034-f2dc-494a-a111-336f52fdaa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:52.626729 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48161034-f2dc-494a-a111-336f52fdaa39 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:52.627017 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48161034-f2dc-494a-a111-336f52fdaa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:52.627294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48161034-f2dc-494a-a111-336f52fdaa39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:52.628046 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48161034-f2dc-494a-a111-336f52fdaa39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:52.629661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48161034-f2dc-494a-a111-336f52fdaa39 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:52.640188 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1410/2847] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:28:52.644516 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436d497-126b-4fe2-a5bb-b10d66b25f40 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:52.655163 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436d497-126b-4fe2-a5bb-b10d66b25f40 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:52.660405 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69add7-24fd-44da-a1c2-996d34c34107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:52.661308 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69add7-24fd-44da-a1c2-996d34c34107 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:52.661585 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69add7-24fd-44da-a1c2-996d34c34107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:52.661858 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69add7-24fd-44da-a1c2-996d34c34107 admin admin] PATH_INFO: `/v3/projects/bd500b202e49473eb1b0c1dd788b3cd4/users/9c4a276bf8644541baa6d42b8a02bb2a/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:52.663994 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69add7-24fd-44da-a1c2-996d34c34107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd500b202e49473eb1b0c1dd788b3cd4, user_id=9c4a276bf8644541baa6d42b8a02bb2a, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:52.666660 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69add7-24fd-44da-a1c2-996d34c34107 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:52.677705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1438/2848] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:28:52 2022] PUT /identity/v3/projects/bd500b202e49473eb1b0c1dd788b3cd4/users/9c4a276bf8644541baa6d42b8a02bb2a/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:52.694192 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0064db1-fbe0-44ab-b76f-66edacaf1507 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:52.694659 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0064db1-fbe0-44ab-b76f-66edacaf1507 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:52.695103 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0064db1-fbe0-44ab-b76f-66edacaf1507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:52.728943 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a0064db1-fbe0-44ab-b76f-66edacaf1507 None None] MFA Rules not processed for user `9c4a276bf8644541baa6d42b8a02bb2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:52.730204 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a0064db1-fbe0-44ab-b76f-66edacaf1507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:52.803355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1411/2849] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:52 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:28:52.814569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69add7-24fd-44da-a1c2-996d34c34107 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:52.816538 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4f69add7-24fd-44da-a1c2-996d34c34107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:52.871450 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:52.876120 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69add7-24fd-44da-a1c2-996d34c34107 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:52.882170 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d3fd2-71f6-4782-a683-6d416e213232 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:52.882828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d3fd2-71f6-4782-a683-6d416e213232 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:52.883115 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d3fd2-71f6-4782-a683-6d416e213232 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:52.883388 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d3fd2-71f6-4782-a683-6d416e213232 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:52.893144 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d3fd2-71f6-4782-a683-6d416e213232 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:52.894660 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d3fd2-71f6-4782-a683-6d416e213232 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:52.904320 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1439/2850] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:28:52 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:28:54.279444 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0064db1-fbe0-44ab-b76f-66edacaf1507 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:54.342802 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0064db1-fbe0-44ab-b76f-66edacaf1507 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:54.348068 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26f9bc-4f30-4297-915c-3981783f6950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:54.348952 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26f9bc-4f30-4297-915c-3981783f6950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:54.349231 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26f9bc-4f30-4297-915c-3981783f6950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:54.349505 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26f9bc-4f30-4297-915c-3981783f6950 admin admin] PATH_INFO: `/v3/users/cae3abcec46c4ebbabba41770b705151` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:54.350747 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26f9bc-4f30-4297-915c-3981783f6950 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cae3abcec46c4ebbabba41770b705151)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:54.352712 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26f9bc-4f30-4297-915c-3981783f6950 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:54.407159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2c26f9bc-4f30-4297-915c-3981783f6950 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cae3abcec46c4ebbabba41770b705151'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:54.407891 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2c26f9bc-4f30-4297-915c-3981783f6950 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cae3abcec46c4ebbabba41770b705151'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:54.408512 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2c26f9bc-4f30-4297-915c-3981783f6950 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cae3abcec46c4ebbabba41770b705151'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:54.421327 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2c26f9bc-4f30-4297-915c-3981783f6950 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cae3abcec46c4ebbabba41770b705151'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:54.422551 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:54.429836 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2c26f9bc-4f30-4297-915c-3981783f6950 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cae3abcec46c4ebbabba41770b705151'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:54.444787 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1412/2851] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:28:54 2022] DELETE /identity/v3/users/cae3abcec46c4ebbabba41770b705151 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:54.449453 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee5b97-3c08-464b-ae47-f86bea3aa053 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:54.450014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee5b97-3c08-464b-ae47-f86bea3aa053 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:54.450014 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee5b97-3c08-464b-ae47-f86bea3aa053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:54.469098 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b0ee5b97-3c08-464b-ae47-f86bea3aa053 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:54.470356 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b0ee5b97-3c08-464b-ae47-f86bea3aa053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:54.531297 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1440/2852] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:28:54.542598 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26f9bc-4f30-4297-915c-3981783f6950 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:54.544281 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2c26f9bc-4f30-4297-915c-3981783f6950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:54.603049 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26f9bc-4f30-4297-915c-3981783f6950 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:54.607355 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b38e13-c259-4975-bab5-a8f5af5714de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:54.608034 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b38e13-c259-4975-bab5-a8f5af5714de service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:54.608310 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b38e13-c259-4975-bab5-a8f5af5714de service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:54.608597 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b38e13-c259-4975-bab5-a8f5af5714de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:54.618177 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b38e13-c259-4975-bab5-a8f5af5714de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:54.619634 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b38e13-c259-4975-bab5-a8f5af5714de service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:54.632293 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1413/2853] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:28:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:28:54.950727 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee5b97-3c08-464b-ae47-f86bea3aa053 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:54.952333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b0ee5b97-3c08-464b-ae47-f86bea3aa053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:54.965912 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee5b97-3c08-464b-ae47-f86bea3aa053 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:54.971762 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d27a7-f5fc-4286-b720-6e00a15f4863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:54.973852 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d27a7-f5fc-4286-b720-6e00a15f4863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:54.974169 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d27a7-f5fc-4286-b720-6e00a15f4863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:54.974555 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d27a7-f5fc-4286-b720-6e00a15f4863 admin admin] PATH_INFO: `/v3/projects/1cda401b9e5e47148522c769b056b47d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:54.976365 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d27a7-f5fc-4286-b720-6e00a15f4863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cda401b9e5e47148522c769b056b47d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:54.978711 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d27a7-f5fc-4286-b720-6e00a15f4863 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:55.032377 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-eb6d27a7-f5fc-4286-b720-6e00a15f4863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cda401b9e5e47148522c769b056b47d', 'request_id': 'req-eb6d27a7-f5fc-4286-b720-6e00a15f4863'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.040964 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-eb6d27a7-f5fc-4286-b720-6e00a15f4863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cda401b9e5e47148522c769b056b47d', 'request_id': 'req-eb6d27a7-f5fc-4286-b720-6e00a15f4863'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.047724 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-eb6d27a7-f5fc-4286-b720-6e00a15f4863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cda401b9e5e47148522c769b056b47d', 'request_id': 'req-eb6d27a7-f5fc-4286-b720-6e00a15f4863'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.059554 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-eb6d27a7-f5fc-4286-b720-6e00a15f4863 admin admin] The token cache is being invalidate because project 1cda401b9e5e47148522c769b056b47d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:28:55.059915 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-eb6d27a7-f5fc-4286-b720-6e00a15f4863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.060580 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-eb6d27a7-f5fc-4286-b720-6e00a15f4863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.062193 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1441/2854] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:28:54 2022] DELETE /identity/v3/projects/1cda401b9e5e47148522c769b056b47d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:55.066279 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b38e13-c259-4975-bab5-a8f5af5714de service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:55.076328 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b38e13-c259-4975-bab5-a8f5af5714de service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:55.137681 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4fe6f-2f74-4611-bc8a-fa3819f88cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:55.138568 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4fe6f-2f74-4611-bc8a-fa3819f88cda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:55.138846 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4fe6f-2f74-4611-bc8a-fa3819f88cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:55.139114 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4fe6f-2f74-4611-bc8a-fa3819f88cda admin admin] PATH_INFO: `/v3/users/9c4a276bf8644541baa6d42b8a02bb2a` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:55.140346 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4fe6f-2f74-4611-bc8a-fa3819f88cda admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c4a276bf8644541baa6d42b8a02bb2a)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:55.142351 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4fe6f-2f74-4611-bc8a-fa3819f88cda admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:55.191490 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a0f4fe6f-2f74-4611-bc8a-fa3819f88cda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c4a276bf8644541baa6d42b8a02bb2a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.192232 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a0f4fe6f-2f74-4611-bc8a-fa3819f88cda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c4a276bf8644541baa6d42b8a02bb2a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.192879 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a0f4fe6f-2f74-4611-bc8a-fa3819f88cda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c4a276bf8644541baa6d42b8a02bb2a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.205649 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a0f4fe6f-2f74-4611-bc8a-fa3819f88cda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c4a276bf8644541baa6d42b8a02bb2a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.213236 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a0f4fe6f-2f74-4611-bc8a-fa3819f88cda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c4a276bf8644541baa6d42b8a02bb2a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.228459 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1414/2855] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:28:55 2022] DELETE /identity/v3/users/9c4a276bf8644541baa6d42b8a02bb2a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:55.502996 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d27a7-f5fc-4286-b720-6e00a15f4863 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:55.504715 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-eb6d27a7-f5fc-4286-b720-6e00a15f4863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:55.521296 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d27a7-f5fc-4286-b720-6e00a15f4863 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:55.580806 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ac804-5d06-4502-ace3-de31317f5dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:55.581682 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ac804-5d06-4502-ace3-de31317f5dcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:55.581963 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ac804-5d06-4502-ace3-de31317f5dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:55.582266 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ac804-5d06-4502-ace3-de31317f5dcc admin admin] PATH_INFO: `/v3/projects/bd500b202e49473eb1b0c1dd788b3cd4` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:55.583493 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ac804-5d06-4502-ace3-de31317f5dcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd500b202e49473eb1b0c1dd788b3cd4)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:55.585452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ac804-5d06-4502-ace3-de31317f5dcc admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:55.602271 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:55.636614 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1d6ac804-5d06-4502-ace3-de31317f5dcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd500b202e49473eb1b0c1dd788b3cd4', 'request_id': 'req-1d6ac804-5d06-4502-ace3-de31317f5dcc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.645343 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1d6ac804-5d06-4502-ace3-de31317f5dcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd500b202e49473eb1b0c1dd788b3cd4', 'request_id': 'req-1d6ac804-5d06-4502-ace3-de31317f5dcc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.651888 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1d6ac804-5d06-4502-ace3-de31317f5dcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd500b202e49473eb1b0c1dd788b3cd4', 'request_id': 'req-1d6ac804-5d06-4502-ace3-de31317f5dcc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.663286 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1d6ac804-5d06-4502-ace3-de31317f5dcc admin admin] The token cache is being invalidate because project bd500b202e49473eb1b0c1dd788b3cd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:28:55.663577 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1d6ac804-5d06-4502-ace3-de31317f5dcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.664268 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1d6ac804-5d06-4502-ace3-de31317f5dcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.665900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1442/2856] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:28:55 2022] DELETE /identity/v3/projects/bd500b202e49473eb1b0c1dd788b3cd4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:55.670950 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9af50-4c36-4757-90a0-dcd9b37d59bc None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:55.671424 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9af50-4c36-4757-90a0-dcd9b37d59bc None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:55.671538 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9af50-4c36-4757-90a0-dcd9b37d59bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:55.679079 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:55.690733 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-86f9af50-4c36-4757-90a0-dcd9b37d59bc None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:55.692530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-86f9af50-4c36-4757-90a0-dcd9b37d59bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:55.751451 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1415/2857] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:28:55.759973 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b532ad8-9424-4dfa-9270-c1d6be1f89af None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:55.760260 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b532ad8-9424-4dfa-9270-c1d6be1f89af None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:55.760539 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b532ad8-9424-4dfa-9270-c1d6be1f89af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:55.779150 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2b532ad8-9424-4dfa-9270-c1d6be1f89af None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:55.780519 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2b532ad8-9424-4dfa-9270-c1d6be1f89af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:55.789046 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1443/2858] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:28:55.793449 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9af50-4c36-4757-90a0-dcd9b37d59bc None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:55.804266 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9af50-4c36-4757-90a0-dcd9b37d59bc None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:55.809367 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703aeed3-eccb-424f-b3fe-b34ba6828752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:55.810060 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703aeed3-eccb-424f-b3fe-b34ba6828752 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:55.810343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703aeed3-eccb-424f-b3fe-b34ba6828752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:55.810668 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703aeed3-eccb-424f-b3fe-b34ba6828752 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:55.811343 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703aeed3-eccb-424f-b3fe-b34ba6828752 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:28:55.811681 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703aeed3-eccb-424f-b3fe-b34ba6828752 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:55.813298 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703aeed3-eccb-424f-b3fe-b34ba6828752 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:55.825455 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1416/2859] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:28:55.829809 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b532ad8-9424-4dfa-9270-c1d6be1f89af None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:55.839969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b532ad8-9424-4dfa-9270-c1d6be1f89af None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:55.845272 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2afed-d054-4cf8-84c7-dc9ab627f36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:55.845940 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2afed-d054-4cf8-84c7-dc9ab627f36b admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:55.846223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2afed-d054-4cf8-84c7-dc9ab627f36b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:55.846496 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2afed-d054-4cf8-84c7-dc9ab627f36b admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:55.847190 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2afed-d054-4cf8-84c7-dc9ab627f36b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:55.849243 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2afed-d054-4cf8-84c7-dc9ab627f36b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:55.879111 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1444/2860] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:28:55 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:28:55.883813 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703aeed3-eccb-424f-b3fe-b34ba6828752 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:55.893934 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703aeed3-eccb-424f-b3fe-b34ba6828752 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:55.954543 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16860758-c233-49b6-a4de-bbd27ebbb810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:55.955213 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16860758-c233-49b6-a4de-bbd27ebbb810 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:55.955494 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16860758-c233-49b6-a4de-bbd27ebbb810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:55.955826 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16860758-c233-49b6-a4de-bbd27ebbb810 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:55.956515 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16860758-c233-49b6-a4de-bbd27ebbb810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:55.958625 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16860758-c233-49b6-a4de-bbd27ebbb810 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:55.979517 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1417/2861] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:28:55 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:28:55.983960 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2afed-d054-4cf8-84c7-dc9ab627f36b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:55.995132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2afed-d054-4cf8-84c7-dc9ab627f36b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:56.000020 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d75ee-454b-4c63-afa2-6b53572c6038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:56.000690 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d75ee-454b-4c63-afa2-6b53572c6038 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:56.000969 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d75ee-454b-4c63-afa2-6b53572c6038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:56.001268 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d75ee-454b-4c63-afa2-6b53572c6038 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:56.002050 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d75ee-454b-4c63-afa2-6b53572c6038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:56.003738 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d75ee-454b-4c63-afa2-6b53572c6038 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:56.015203 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1445/2862] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:28:56.019474 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16860758-c233-49b6-a4de-bbd27ebbb810 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:56.029672 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16860758-c233-49b6-a4de-bbd27ebbb810 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:56.034500 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd1702-f569-41d9-9d63-cb289d0edfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:56.035141 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd1702-f569-41d9-9d63-cb289d0edfa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:56.035419 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd1702-f569-41d9-9d63-cb289d0edfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:56.035733 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd1702-f569-41d9-9d63-cb289d0edfa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:56.036544 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd1702-f569-41d9-9d63-cb289d0edfa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:56.038180 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd1702-f569-41d9-9d63-cb289d0edfa6 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:56.048648 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1418/2863] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:28:56.052755 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d75ee-454b-4c63-afa2-6b53572c6038 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:56.063295 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d75ee-454b-4c63-afa2-6b53572c6038 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:56.068873 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c23faa-9d11-42d3-83ca-34e62734b18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:56.069762 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c23faa-9d11-42d3-83ca-34e62734b18a admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:56.070064 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c23faa-9d11-42d3-83ca-34e62734b18a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:56.070316 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c23faa-9d11-42d3-83ca-34e62734b18a admin admin] PATH_INFO: `/v3/projects/5496b021ccb8458ca1cbd18dfe990b12/users/76f22fe8fcde4396b0e41fa7d723c85f/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:56.085194 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c23faa-9d11-42d3-83ca-34e62734b18a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5496b021ccb8458ca1cbd18dfe990b12, user_id=76f22fe8fcde4396b0e41fa7d723c85f, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:56.087882 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c23faa-9d11-42d3-83ca-34e62734b18a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:56.103054 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1446/2864] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:28:56 2022] PUT /identity/v3/projects/5496b021ccb8458ca1cbd18dfe990b12/users/76f22fe8fcde4396b0e41fa7d723c85f/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:56.167745 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fda39-4736-4e80-906e-d6aff1cbf72e None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:56.168020 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fda39-4736-4e80-906e-d6aff1cbf72e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:56.168290 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fda39-4736-4e80-906e-d6aff1cbf72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:56.197249 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-331fda39-4736-4e80-906e-d6aff1cbf72e None None] MFA Rules not processed for user `76f22fe8fcde4396b0e41fa7d723c85f`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:56.198462 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-331fda39-4736-4e80-906e-d6aff1cbf72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:56.261918 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:56.265757 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1419/2865] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:28:56.270823 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c23faa-9d11-42d3-83ca-34e62734b18a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:56.339683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c23faa-9d11-42d3-83ca-34e62734b18a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:56.345008 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388720c-fba1-4bff-a6ea-0b699ac470fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:56.345683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388720c-fba1-4bff-a6ea-0b699ac470fb admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:56.345966 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388720c-fba1-4bff-a6ea-0b699ac470fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:56.346243 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388720c-fba1-4bff-a6ea-0b699ac470fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:56.346936 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388720c-fba1-4bff-a6ea-0b699ac470fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:56.348910 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388720c-fba1-4bff-a6ea-0b699ac470fb admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:56.376598 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1447/2866] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:28:56 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:28:56.381278 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331fda39-4736-4e80-906e-d6aff1cbf72e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:56.392706 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331fda39-4736-4e80-906e-d6aff1cbf72e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:56.453314 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda4e25-7b2f-4e8e-a45a-f048ef505e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:56.454248 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda4e25-7b2f-4e8e-a45a-f048ef505e44 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:56.454634 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda4e25-7b2f-4e8e-a45a-f048ef505e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:56.454988 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda4e25-7b2f-4e8e-a45a-f048ef505e44 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:56.456100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda4e25-7b2f-4e8e-a45a-f048ef505e44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:56.458872 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda4e25-7b2f-4e8e-a45a-f048ef505e44 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:56.480217 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1420/2867] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:28:56 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:28:56.484391 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388720c-fba1-4bff-a6ea-0b699ac470fb admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:56.495281 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388720c-fba1-4bff-a6ea-0b699ac470fb admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:56.500992 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b07c9f-0497-4ef5-8f6a-f7ddd4adac02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:56.501663 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b07c9f-0497-4ef5-8f6a-f7ddd4adac02 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:56.501950 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b07c9f-0497-4ef5-8f6a-f7ddd4adac02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:56.502224 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b07c9f-0497-4ef5-8f6a-f7ddd4adac02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:56.502958 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b07c9f-0497-4ef5-8f6a-f7ddd4adac02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:56.504624 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b07c9f-0497-4ef5-8f6a-f7ddd4adac02 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:56.516236 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1448/2868] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:28:56.520321 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda4e25-7b2f-4e8e-a45a-f048ef505e44 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:56.530968 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda4e25-7b2f-4e8e-a45a-f048ef505e44 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:56.536235 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421fe8a-0bcf-42f9-9731-c93ec608acad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:56.537153 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421fe8a-0bcf-42f9-9731-c93ec608acad admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:56.537435 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421fe8a-0bcf-42f9-9731-c93ec608acad admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:56.537728 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421fe8a-0bcf-42f9-9731-c93ec608acad admin admin] PATH_INFO: `/v3/projects/ad566d6c5e194045b2216db3ae3b975b/users/6ddb4f0c3f0649bda9c25ef676e950b9/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:56.553140 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9421fe8a-0bcf-42f9-9731-c93ec608acad admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad566d6c5e194045b2216db3ae3b975b, user_id=6ddb4f0c3f0649bda9c25ef676e950b9, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:56.555850 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9421fe8a-0bcf-42f9-9731-c93ec608acad admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:56.570671 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1421/2869] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:28:56 2022] PUT /identity/v3/projects/ad566d6c5e194045b2216db3ae3b975b/users/6ddb4f0c3f0649bda9c25ef676e950b9/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:56.574646 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b07c9f-0497-4ef5-8f6a-f7ddd4adac02 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:56.584828 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b07c9f-0497-4ef5-8f6a-f7ddd4adac02 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:56.597226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:56.643937 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c569157-4cb2-437f-9905-9e1bcbe570d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:56.644606 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c569157-4cb2-437f-9905-9e1bcbe570d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:56.644887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c569157-4cb2-437f-9905-9e1bcbe570d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:56.645165 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c569157-4cb2-437f-9905-9e1bcbe570d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:56.646044 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c569157-4cb2-437f-9905-9e1bcbe570d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:56.647718 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c569157-4cb2-437f-9905-9e1bcbe570d3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:56.658021 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1449/2870] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:28:56.662400 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421fe8a-0bcf-42f9-9731-c93ec608acad admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:56.673736 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421fe8a-0bcf-42f9-9731-c93ec608acad admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:56.679962 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ba263-2fbb-4e08-a803-0f8ecd56041b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:56.680864 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ba263-2fbb-4e08-a803-0f8ecd56041b admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:56.681151 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ba263-2fbb-4e08-a803-0f8ecd56041b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:56.681425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ba263-2fbb-4e08-a803-0f8ecd56041b admin admin] PATH_INFO: `/v3/projects/ad566d6c5e194045b2216db3ae3b975b/users/6ddb4f0c3f0649bda9c25ef676e950b9/roles/5c3c095bb53943eca95c4e049cd8cbf5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:56.683701 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ba263-2fbb-4e08-a803-0f8ecd56041b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad566d6c5e194045b2216db3ae3b975b, user_id=6ddb4f0c3f0649bda9c25ef676e950b9, role_id=5c3c095bb53943eca95c4e049cd8cbf5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:56.686386 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ba263-2fbb-4e08-a803-0f8ecd56041b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:56.697941 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1422/2871] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:28:56 2022] PUT /identity/v3/projects/ad566d6c5e194045b2216db3ae3b975b/users/6ddb4f0c3f0649bda9c25ef676e950b9/roles/5c3c095bb53943eca95c4e049cd8cbf5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:56.708441 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf68f15-d4b7-4486-b3e1-890b6fc8d0de None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:56.708730 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf68f15-d4b7-4486-b3e1-890b6fc8d0de None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:56.709012 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf68f15-d4b7-4486-b3e1-890b6fc8d0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:56.742307 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6cf68f15-d4b7-4486-b3e1-890b6fc8d0de None None] MFA Rules not processed for user `6ddb4f0c3f0649bda9c25ef676e950b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:28:56.744055 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6cf68f15-d4b7-4486-b3e1-890b6fc8d0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:56.831490 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1450/2872] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:28:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:28:56.842601 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ba263-2fbb-4e08-a803-0f8ecd56041b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:56.844682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-bb2ba263-2fbb-4e08-a803-0f8ecd56041b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:56.904996 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ba263-2fbb-4e08-a803-0f8ecd56041b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:56.909450 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fefe6-670b-47a5-989f-7e645fb4cea6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:56.910097 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fefe6-670b-47a5-989f-7e645fb4cea6 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:56.910383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fefe6-670b-47a5-989f-7e645fb4cea6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:56.910649 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fefe6-670b-47a5-989f-7e645fb4cea6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:56.985042 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8fefe6-670b-47a5-989f-7e645fb4cea6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:56.986546 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8fefe6-670b-47a5-989f-7e645fb4cea6 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:56.996998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1423/2873] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:28:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:29:01.196765 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf68f15-d4b7-4486-b3e1-890b6fc8d0de None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:29:01.198378 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6cf68f15-d4b7-4486-b3e1-890b6fc8d0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:01.211257 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf68f15-d4b7-4486-b3e1-890b6fc8d0de None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:29:01.215435 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c3451-747b-49e8-a76b-77f05c746629 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:29:01.216141 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c3451-747b-49e8-a76b-77f05c746629 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:01.216421 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c3451-747b-49e8-a76b-77f05c746629 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:01.216686 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c3451-747b-49e8-a76b-77f05c746629 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:01.226074 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c3451-747b-49e8-a76b-77f05c746629 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:29:01.227577 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c3451-747b-49e8-a76b-77f05c746629 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:29:01.240226 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1451/2874] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:29:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:29:06.160682 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fefe6-670b-47a5-989f-7e645fb4cea6 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:29:06.170587 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:29:06.228281 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fefe6-670b-47a5-989f-7e645fb4cea6 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:29:06.237294 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5b7df-bfc3-4661-9313-02c8b33063c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:29:06.238619 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5b7df-bfc3-4661-9313-02c8b33063c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:06.239002 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5b7df-bfc3-4661-9313-02c8b33063c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:06.239372 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5b7df-bfc3-4661-9313-02c8b33063c2 admin admin] PATH_INFO: `/v3/users/76f22fe8fcde4396b0e41fa7d723c85f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:06.241428 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a5b7df-bfc3-4661-9313-02c8b33063c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76f22fe8fcde4396b0e41fa7d723c85f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:29:06.244173 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a5b7df-bfc3-4661-9313-02c8b33063c2 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:29:06.294983 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-14a5b7df-bfc3-4661-9313-02c8b33063c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76f22fe8fcde4396b0e41fa7d723c85f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:29:06.295661 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-14a5b7df-bfc3-4661-9313-02c8b33063c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76f22fe8fcde4396b0e41fa7d723c85f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:29:06.296274 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-14a5b7df-bfc3-4661-9313-02c8b33063c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76f22fe8fcde4396b0e41fa7d723c85f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:29:06.308377 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-14a5b7df-bfc3-4661-9313-02c8b33063c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76f22fe8fcde4396b0e41fa7d723c85f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:29:06.315799 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-14a5b7df-bfc3-4661-9313-02c8b33063c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76f22fe8fcde4396b0e41fa7d723c85f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:29:06.333331 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1424/2875] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:29:06 2022] DELETE /identity/v3/users/76f22fe8fcde4396b0e41fa7d723c85f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:29:06.340196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f475f1-9fca-4410-9515-1789f558c850 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:06.340937 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f475f1-9fca-4410-9515-1789f558c850 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:06.341521 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f475f1-9fca-4410-9515-1789f558c850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:06.366113 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-17f475f1-9fca-4410-9515-1789f558c850 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:29:06.367371 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-17f475f1-9fca-4410-9515-1789f558c850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:06.433554 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1452/2876] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:29:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:29:06.445487 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5b7df-bfc3-4661-9313-02c8b33063c2 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:29:06.447321 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-14a5b7df-bfc3-4661-9313-02c8b33063c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:06.506786 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5b7df-bfc3-4661-9313-02c8b33063c2 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:29:06.510948 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fdb4a-211a-4b21-a39e-5b83c9a7b748 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:29:06.511641 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fdb4a-211a-4b21-a39e-5b83c9a7b748 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:06.511939 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fdb4a-211a-4b21-a39e-5b83c9a7b748 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:06.512213 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fdb4a-211a-4b21-a39e-5b83c9a7b748 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:06.521892 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5fdb4a-211a-4b21-a39e-5b83c9a7b748 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:29:06.523345 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5fdb4a-211a-4b21-a39e-5b83c9a7b748 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:29:06.533090 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1425/2877] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:29:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:29:06.800092 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f475f1-9fca-4410-9515-1789f558c850 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:29:06.801706 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-17f475f1-9fca-4410-9515-1789f558c850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:06.815288 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f475f1-9fca-4410-9515-1789f558c850 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:29:06.820976 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70b17c-8f58-4a73-a59f-f26e6a50ece5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:29:06.822045 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70b17c-8f58-4a73-a59f-f26e6a50ece5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:06.822349 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70b17c-8f58-4a73-a59f-f26e6a50ece5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:06.822621 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70b17c-8f58-4a73-a59f-f26e6a50ece5 admin admin] PATH_INFO: `/v3/projects/5496b021ccb8458ca1cbd18dfe990b12` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:06.824114 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70b17c-8f58-4a73-a59f-f26e6a50ece5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5496b021ccb8458ca1cbd18dfe990b12)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:29:06.825972 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70b17c-8f58-4a73-a59f-f26e6a50ece5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:29:06.848259 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:29:06.874569 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9f70b17c-8f58-4a73-a59f-f26e6a50ece5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5496b021ccb8458ca1cbd18dfe990b12', 'request_id': 'req-9f70b17c-8f58-4a73-a59f-f26e6a50ece5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:29:06.883388 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9f70b17c-8f58-4a73-a59f-f26e6a50ece5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5496b021ccb8458ca1cbd18dfe990b12', 'request_id': 'req-9f70b17c-8f58-4a73-a59f-f26e6a50ece5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:29:06.890013 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9f70b17c-8f58-4a73-a59f-f26e6a50ece5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5496b021ccb8458ca1cbd18dfe990b12', 'request_id': 'req-9f70b17c-8f58-4a73-a59f-f26e6a50ece5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:29:06.902662 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9f70b17c-8f58-4a73-a59f-f26e6a50ece5 admin admin] The token cache is being invalidate because project 5496b021ccb8458ca1cbd18dfe990b12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:29:06.902949 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9f70b17c-8f58-4a73-a59f-f26e6a50ece5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:29:06.903687 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9f70b17c-8f58-4a73-a59f-f26e6a50ece5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:29:06.905344 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1453/2878] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:29:06 2022] DELETE /identity/v3/projects/5496b021ccb8458ca1cbd18dfe990b12 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:29:06.909396 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fdb4a-211a-4b21-a39e-5b83c9a7b748 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:29:06.919508 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fdb4a-211a-4b21-a39e-5b83c9a7b748 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:29:06.946686 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:29:06.978978 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18decfde-2f05-444d-be44-9693094ee9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:29:06.979982 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18decfde-2f05-444d-be44-9693094ee9dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:06.980260 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18decfde-2f05-444d-be44-9693094ee9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:06.980526 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18decfde-2f05-444d-be44-9693094ee9dd admin admin] PATH_INFO: `/v3/users/6ddb4f0c3f0649bda9c25ef676e950b9` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:06.981739 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18decfde-2f05-444d-be44-9693094ee9dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ddb4f0c3f0649bda9c25ef676e950b9)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:29:06.983683 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18decfde-2f05-444d-be44-9693094ee9dd admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:29:07.034440 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-18decfde-2f05-444d-be44-9693094ee9dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ddb4f0c3f0649bda9c25ef676e950b9'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:29:07.035195 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-18decfde-2f05-444d-be44-9693094ee9dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ddb4f0c3f0649bda9c25ef676e950b9'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:29:07.035964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-18decfde-2f05-444d-be44-9693094ee9dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ddb4f0c3f0649bda9c25ef676e950b9'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:29:07.053638 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-18decfde-2f05-444d-be44-9693094ee9dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ddb4f0c3f0649bda9c25ef676e950b9'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:29:07.062273 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-18decfde-2f05-444d-be44-9693094ee9dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ddb4f0c3f0649bda9c25ef676e950b9'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:29:07.077500 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1426/2879] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:29:06 2022] DELETE /identity/v3/users/6ddb4f0c3f0649bda9c25ef676e950b9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:29:07.353903 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70b17c-8f58-4a73-a59f-f26e6a50ece5 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:29:07.355580 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9f70b17c-8f58-4a73-a59f-f26e6a50ece5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:07.370815 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70b17c-8f58-4a73-a59f-f26e6a50ece5 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:29:07.434149 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66457aaa-3604-43c1-a265-7c300af85fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:29:07.435038 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66457aaa-3604-43c1-a265-7c300af85fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:07.435334 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66457aaa-3604-43c1-a265-7c300af85fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:07.435594 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66457aaa-3604-43c1-a265-7c300af85fb7 admin admin] PATH_INFO: `/v3/projects/ad566d6c5e194045b2216db3ae3b975b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:07.436840 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66457aaa-3604-43c1-a265-7c300af85fb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad566d6c5e194045b2216db3ae3b975b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:29:07.438771 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66457aaa-3604-43c1-a265-7c300af85fb7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:29:07.486248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-66457aaa-3604-43c1-a265-7c300af85fb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad566d6c5e194045b2216db3ae3b975b', 'request_id': 'req-66457aaa-3604-43c1-a265-7c300af85fb7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:29:07.494323 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-66457aaa-3604-43c1-a265-7c300af85fb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad566d6c5e194045b2216db3ae3b975b', 'request_id': 'req-66457aaa-3604-43c1-a265-7c300af85fb7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:29:07.500001 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-66457aaa-3604-43c1-a265-7c300af85fb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad566d6c5e194045b2216db3ae3b975b', 'request_id': 'req-66457aaa-3604-43c1-a265-7c300af85fb7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:29:07.511703 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-66457aaa-3604-43c1-a265-7c300af85fb7 admin admin] The token cache is being invalidate because project ad566d6c5e194045b2216db3ae3b975b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:29:07.511994 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-66457aaa-3604-43c1-a265-7c300af85fb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:29:07.512692 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-66457aaa-3604-43c1-a265-7c300af85fb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:29:07.514333 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1454/2880] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:29:07 2022] DELETE /identity/v3/projects/ad566d6c5e194045b2216db3ae3b975b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:29:07.520477 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f6b18-42ce-4356-b048-363f592fff09 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:07.520987 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f6b18-42ce-4356-b048-363f592fff09 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:07.521100 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f6b18-42ce-4356-b048-363f592fff09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:07.550399 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1a0f6b18-42ce-4356-b048-363f592fff09 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:29:07.551632 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1a0f6b18-42ce-4356-b048-363f592fff09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:07.616060 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1427/2881] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:29:07.625567 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca08c9-680b-4f2c-847b-a49f4b0533c9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:07.625854 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca08c9-680b-4f2c-847b-a49f4b0533c9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:07.626130 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca08c9-680b-4f2c-847b-a49f4b0533c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:07.646076 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3dca08c9-680b-4f2c-847b-a49f4b0533c9 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:29:07.647277 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3dca08c9-680b-4f2c-847b-a49f4b0533c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:07.656196 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1455/2882] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:29:07.660877 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f6b18-42ce-4356-b048-363f592fff09 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:29:07.671486 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f6b18-42ce-4356-b048-363f592fff09 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:29:07.676754 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7b61f-4149-47b9-a20c-5a569dcfa7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:29:07.677445 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7b61f-4149-47b9-a20c-5a569dcfa7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:07.677739 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7b61f-4149-47b9-a20c-5a569dcfa7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:07.678016 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7b61f-4149-47b9-a20c-5a569dcfa7b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:07.678753 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7b61f-4149-47b9-a20c-5a569dcfa7b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:29:07.679063 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7b61f-4149-47b9-a20c-5a569dcfa7b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:29:07.680861 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7b61f-4149-47b9-a20c-5a569dcfa7b4 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:29:07.694209 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1428/2883] 10.222.0.27 () {58 vars in 1152 bytes} [Mon Jun 27 21:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:29:07.699543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca08c9-680b-4f2c-847b-a49f4b0533c9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:29:07.713513 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca08c9-680b-4f2c-847b-a49f4b0533c9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:29:07.721280 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4b0d2-d20e-42c6-a67f-9315e6b0ba61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:29:07.721936 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4b0d2-d20e-42c6-a67f-9315e6b0ba61 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:07.722213 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4b0d2-d20e-42c6-a67f-9315e6b0ba61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:07.722485 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4b0d2-d20e-42c6-a67f-9315e6b0ba61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:07.723175 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4b0d2-d20e-42c6-a67f-9315e6b0ba61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:29:07.725140 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4b0d2-d20e-42c6-a67f-9315e6b0ba61 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:29:07.757803 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1456/2884] 10.222.0.27 () {60 vars in 1152 bytes} [Mon Jun 27 21:29:07 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:29:07.762105 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7b61f-4149-47b9-a20c-5a569dcfa7b4 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:29:07.772497 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7b61f-4149-47b9-a20c-5a569dcfa7b4 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:29:07.797181 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:29:07.831169 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88db16-b935-4556-a03c-208d634b5694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:29:07.831873 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88db16-b935-4556-a03c-208d634b5694 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:07.832159 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88db16-b935-4556-a03c-208d634b5694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:07.832432 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88db16-b935-4556-a03c-208d634b5694 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:07.833104 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88db16-b935-4556-a03c-208d634b5694 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:29:07.835019 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88db16-b935-4556-a03c-208d634b5694 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:29:07.856680 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1429/2885] 10.222.0.27 () {60 vars in 1143 bytes} [Mon Jun 27 21:29:07 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:29:07.861225 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4b0d2-d20e-42c6-a67f-9315e6b0ba61 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:29:07.872179 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4b0d2-d20e-42c6-a67f-9315e6b0ba61 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:29:07.878447 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75b820-f548-4fb3-9ef4-6b33ea93d5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:29:07.879114 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75b820-f548-4fb3-9ef4-6b33ea93d5f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:07.879380 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75b820-f548-4fb3-9ef4-6b33ea93d5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:07.879683 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75b820-f548-4fb3-9ef4-6b33ea93d5f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:07.880412 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75b820-f548-4fb3-9ef4-6b33ea93d5f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:29:07.882443 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75b820-f548-4fb3-9ef4-6b33ea93d5f2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:29:07.884746 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:29:07.894757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1457/2886] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:29:07.899284 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88db16-b935-4556-a03c-208d634b5694 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:29:07.909994 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88db16-b935-4556-a03c-208d634b5694 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:29:07.917530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa2cfd-22fd-4eb0-9ee2-490cfc900459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:29:07.918206 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa2cfd-22fd-4eb0-9ee2-490cfc900459 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:07.918498 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa2cfd-22fd-4eb0-9ee2-490cfc900459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:07.918777 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa2cfd-22fd-4eb0-9ee2-490cfc900459 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:07.919563 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa2cfd-22fd-4eb0-9ee2-490cfc900459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:29:07.921219 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa2cfd-22fd-4eb0-9ee2-490cfc900459 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:29:07.932383 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1430/2887] 10.222.0.27 () {58 vars in 1121 bytes} [Mon Jun 27 21:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:29:07.936613 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75b820-f548-4fb3-9ef4-6b33ea93d5f2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:29:07.947543 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75b820-f548-4fb3-9ef4-6b33ea93d5f2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:29:07.953337 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973dd3a4-4e01-40bf-9449-b868c7146e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:29:07.954230 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dd3a4-4e01-40bf-9449-b868c7146e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:07.954506 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dd3a4-4e01-40bf-9449-b868c7146e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:07.954761 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dd3a4-4e01-40bf-9449-b868c7146e74 admin admin] PATH_INFO: `/v3/projects/ef69c081a67644c7aecdcc06fc8e8675/users/836fc578997845b898bdc310ff93ca37/roles/b1bcb7445ee54636ae7b29a2485cc201` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:07.970215 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973dd3a4-4e01-40bf-9449-b868c7146e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef69c081a67644c7aecdcc06fc8e8675, user_id=836fc578997845b898bdc310ff93ca37, role_id=b1bcb7445ee54636ae7b29a2485cc201)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:29:07.972938 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973dd3a4-4e01-40bf-9449-b868c7146e74 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:29:07.990154 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1458/2888] 10.222.0.27 () {60 vars in 1482 bytes} [Mon Jun 27 21:29:07 2022] PUT /identity/v3/projects/ef69c081a67644c7aecdcc06fc8e8675/users/836fc578997845b898bdc310ff93ca37/roles/b1bcb7445ee54636ae7b29a2485cc201 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:29:07.995285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefd3f8-250a-437a-880a-3b4a288e3ff0 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:07.995577 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefd3f8-250a-437a-880a-3b4a288e3ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:07.995887 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefd3f8-250a-437a-880a-3b4a288e3ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:08.016425 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-dcefd3f8-250a-437a-880a-3b4a288e3ff0 None None] MFA Rules not processed for user `3dc250d3d60a40d8b744430d137fcb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:29:08.017634 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-dcefd3f8-250a-437a-880a-3b4a288e3ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:08.086333 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1431/2889] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:29:08.096185 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973dd3a4-4e01-40bf-9449-b868c7146e74 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:29:08.098123 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-973dd3a4-4e01-40bf-9449-b868c7146e74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:08.164715 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973dd3a4-4e01-40bf-9449-b868c7146e74 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:29:08.169589 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e036f4-deba-486c-95f9-45acc131f40f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:29:08.170242 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e036f4-deba-486c-95f9-45acc131f40f service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:08.170527 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e036f4-deba-486c-95f9-45acc131f40f service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:08.170795 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e036f4-deba-486c-95f9-45acc131f40f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:08.180384 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e036f4-deba-486c-95f9-45acc131f40f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:29:08.181917 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e036f4-deba-486c-95f9-45acc131f40f service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:29:08.191332 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1459/2890] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:29:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:29:08.799347 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8a95d-457c-41d4-a3f0-9295e8afb058 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:08.799864 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8a95d-457c-41d4-a3f0-9295e8afb058 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:08.799978 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8a95d-457c-41d4-a3f0-9295e8afb058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:08.828275 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-59d8a95d-457c-41d4-a3f0-9295e8afb058 None None] MFA Rules not processed for user `836fc578997845b898bdc310ff93ca37`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:29:08.829482 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-59d8a95d-457c-41d4-a3f0-9295e8afb058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:08.900550 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1432/2891] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:29:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:29:08.911705 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e036f4-deba-486c-95f9-45acc131f40f service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:29:08.913223 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c0e036f4-deba-486c-95f9-45acc131f40f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:08.975866 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e036f4-deba-486c-95f9-45acc131f40f service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:29:08.979477 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5704d0c-3283-4666-a562-8588fb5e4b1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16fda36445d9415aae4df74f31af0601', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:29:08.980177 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5704d0c-3283-4666-a562-8588fb5e4b1d service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:08.980453 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5704d0c-3283-4666-a562-8588fb5e4b1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:08.980727 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5704d0c-3283-4666-a562-8588fb5e4b1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:08.989869 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5704d0c-3283-4666-a562-8588fb5e4b1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:29:08.991337 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5704d0c-3283-4666-a562-8588fb5e4b1d service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:29:09.000790 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1460/2892] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:29:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:29:10.371985 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d73a82-af45-4c9c-8ce0-f00e501b9c69 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:10.372530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d73a82-af45-4c9c-8ce0-f00e501b9c69 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:10.372530 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d73a82-af45-4c9c-8ce0-f00e501b9c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:10.392507 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-c4d73a82-af45-4c9c-8ce0-f00e501b9c69 None None] MFA Rules not processed for user `836fc578997845b898bdc310ff93ca37`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:29:10.393743 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c4d73a82-af45-4c9c-8ce0-f00e501b9c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:10.402012 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1433/2893] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:29:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:29:10.644452 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5704d0c-3283-4666-a562-8588fb5e4b1d service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:29:10.646237 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c5704d0c-3283-4666-a562-8588fb5e4b1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:10.660659 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5704d0c-3283-4666-a562-8588fb5e4b1d service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:29:10.665264 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c030bf-355d-48ff-8ecc-2a1f5a41e28d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c7ce29db6b14176a4c9be436bcddc6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:29:10.665900 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c030bf-355d-48ff-8ecc-2a1f5a41e28d service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:10.666183 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c030bf-355d-48ff-8ecc-2a1f5a41e28d service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:10.666502 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c030bf-355d-48ff-8ecc-2a1f5a41e28d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:10.668596 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-05c030bf-355d-48ff-8ecc-2a1f5a41e28d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:10.680193 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c030bf-355d-48ff-8ecc-2a1f5a41e28d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:29:10.681660 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c030bf-355d-48ff-8ecc-2a1f5a41e28d service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:29:10.691092 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1461/2894] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:29:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:29:20.079693 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d73a82-af45-4c9c-8ce0-f00e501b9c69 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:29:20.081266 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c4d73a82-af45-4c9c-8ce0-f00e501b9c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:20.114863 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:29:20.190098 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d73a82-af45-4c9c-8ce0-f00e501b9c69 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:29:20.196081 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec7fca-7b17-4b75-bc43-c24c37f3c226 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7fe01ef64f46f6a0e77d5945219b33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:29:20.196717 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec7fca-7b17-4b75-bc43-c24c37f3c226 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:20.196995 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec7fca-7b17-4b75-bc43-c24c37f3c226 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:20.197268 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec7fca-7b17-4b75-bc43-c24c37f3c226 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:20.199128 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f1ec7fca-7b17-4b75-bc43-c24c37f3c226 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:20.211335 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec7fca-7b17-4b75-bc43-c24c37f3c226 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:29:20.212950 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec7fca-7b17-4b75-bc43-c24c37f3c226 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:29:20.223626 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1434/2895] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:29:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:29:20.686891 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=69,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:29:25.750752 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c030bf-355d-48ff-8ecc-2a1f5a41e28d service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:29:25.752564 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-05c030bf-355d-48ff-8ecc-2a1f5a41e28d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:25.847855 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c030bf-355d-48ff-8ecc-2a1f5a41e28d service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:29:25.856132 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068108f2-1720-4303-862a-b92639f5e464 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5fa9b0b0314ee9bb7e16b8464307b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:29:25.857145 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068108f2-1720-4303-862a-b92639f5e464 service placement] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:25.857511 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068108f2-1720-4303-862a-b92639f5e464 service placement] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:25.857805 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068108f2-1720-4303-862a-b92639f5e464 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:25.861102 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-068108f2-1720-4303-862a-b92639f5e464 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:25.878500 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068108f2-1720-4303-862a-b92639f5e464 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:29:25.880161 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068108f2-1720-4303-862a-b92639f5e464 service placement] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:29:25.891138 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1462/2896] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:29:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:29:27.044420 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b55ad-f58b-4f42-a7d0-ac0ad169aaa6 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:27.045007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b55ad-f58b-4f42-a7d0-ac0ad169aaa6 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:27.045007 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b55ad-f58b-4f42-a7d0-ac0ad169aaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:27.063927 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-4a8b55ad-f58b-4f42-a7d0-ac0ad169aaa6 None None] MFA Rules not processed for user `836fc578997845b898bdc310ff93ca37`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:29:27.065133 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4a8b55ad-f58b-4f42-a7d0-ac0ad169aaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:27.073465 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1435/2897] 10.222.0.27 () {58 vars in 957 bytes} [Mon Jun 27 21:29:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:29:35.886706 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=41 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:29:37.058998 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=36 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:29:49.626811 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ab818-0fb2-4ac2-8c70-e0c39f85bd9a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:49.627363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ab818-0fb2-4ac2-8c70-e0c39f85bd9a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:49.627363 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ab818-0fb2-4ac2-8c70-e0c39f85bd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:49.691289 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b11ab818-0fb2-4ac2-8c70-e0c39f85bd9a None None] MFA Rules not processed for user `e802da0b078e457da466a9a7b0f15085`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:29:49.692560 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b11ab818-0fb2-4ac2-8c70-e0c39f85bd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:49.769928 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1463/2898] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Jun 27 21:29:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:29:49.817862 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b55ad-f58b-4f42-a7d0-ac0ad169aaa6 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:29:49.819432 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4a8b55ad-f58b-4f42-a7d0-ac0ad169aaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:49.909041 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b55ad-f58b-4f42-a7d0-ac0ad169aaa6 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:29:49.915158 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d626f-47ff-4c14-bdb5-79dd97ec34a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28acb0322114aebafcf62bdf9a346df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:29:49.915853 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d626f-47ff-4c14-bdb5-79dd97ec34a0 service swift] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:49.916142 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d626f-47ff-4c14-bdb5-79dd97ec34a0 service swift] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:49.916421 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d626f-47ff-4c14-bdb5-79dd97ec34a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:49.925259 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d626f-47ff-4c14-bdb5-79dd97ec34a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:29:49.926864 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d626f-47ff-4c14-bdb5-79dd97ec34a0 service swift] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:29:49.935320 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1436/2899] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Jun 27 21:29:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:29:59.766490 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=44 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:29:59.931964 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=36 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:30:05.257576 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7e40a-053b-4224-9e70-7eef344b625e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:30:05.258424 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7e40a-053b-4224-9e70-7eef344b625e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:30:05.258537 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7e40a-053b-4224-9e70-7eef344b625e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:30:05.281681 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e2f7e40a-053b-4224-9e70-7eef344b625e None None] MFA Rules not processed for user `e802da0b078e457da466a9a7b0f15085`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:30:05.282988 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e2f7e40a-053b-4224-9e70-7eef344b625e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:30:05.290260 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1464/2900] 10.222.0.27 () {58 vars in 967 bytes} [Mon Jun 27 21:30:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:30:05.302532 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d626f-47ff-4c14-bdb5-79dd97ec34a0 service swift] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:30:05.313760 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d626f-47ff-4c14-bdb5-79dd97ec34a0 service swift] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:30:05.318945 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6629e1-d239-4ed9-8ebe-7b42f5f1b223 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f28acb0322114aebafcf62bdf9a346df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:30:05.319598 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6629e1-d239-4ed9-8ebe-7b42f5f1b223 service swift] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:30:05.319918 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6629e1-d239-4ed9-8ebe-7b42f5f1b223 service swift] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:30:05.320226 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6629e1-d239-4ed9-8ebe-7b42f5f1b223 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:30:05.328497 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6629e1-d239-4ed9-8ebe-7b42f5f1b223 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:30:05.330106 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6629e1-d239-4ed9-8ebe-7b42f5f1b223 service swift] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:30:05.338125 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1437/2901] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Jun 27 21:30:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:30:07.300116 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7e40a-053b-4224-9e70-7eef344b625e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:30:07.314323 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7e40a-053b-4224-9e70-7eef344b625e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:30:07.319242 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4ce9b-9684-413d-9129-587ed0450c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:30:07.320195 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4ce9b-9684-413d-9129-587ed0450c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:30:07.320479 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4ce9b-9684-413d-9129-587ed0450c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:30:07.320757 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4ce9b-9684-413d-9129-587ed0450c4c admin admin] PATH_INFO: `/v3/users/836fc578997845b898bdc310ff93ca37` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:30:07.321976 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4ce9b-9684-413d-9129-587ed0450c4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=836fc578997845b898bdc310ff93ca37)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:30:07.323993 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4ce9b-9684-413d-9129-587ed0450c4c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:30:07.372376 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-daa4ce9b-9684-413d-9129-587ed0450c4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '836fc578997845b898bdc310ff93ca37'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:30:07.373001 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-daa4ce9b-9684-413d-9129-587ed0450c4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '836fc578997845b898bdc310ff93ca37'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:30:07.373609 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-daa4ce9b-9684-413d-9129-587ed0450c4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '836fc578997845b898bdc310ff93ca37'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:30:07.385996 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-daa4ce9b-9684-413d-9129-587ed0450c4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '836fc578997845b898bdc310ff93ca37'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:30:07.392888 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-daa4ce9b-9684-413d-9129-587ed0450c4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '836fc578997845b898bdc310ff93ca37'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:30:07.407187 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1465/2902] 10.222.0.27 () {58 vars in 1223 bytes} [Mon Jun 27 21:30:07 2022] DELETE /identity/v3/users/836fc578997845b898bdc310ff93ca37 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:30:07.589387 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6629e1-d239-4ed9-8ebe-7b42f5f1b223 service swift] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:30:07.600285 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6629e1-d239-4ed9-8ebe-7b42f5f1b223 service swift] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:30:07.659540 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020337c-ff70-4f84-8870-207dba3e33fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dc250d3d60a40d8b744430d137fcb9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2213409bed61490ba55498fcd7bd10f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:30:07.660458 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020337c-ff70-4f84-8870-207dba3e33fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:30:07.660741 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020337c-ff70-4f84-8870-207dba3e33fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:30:07.661016 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020337c-ff70-4f84-8870-207dba3e33fe admin admin] PATH_INFO: `/v3/projects/ef69c081a67644c7aecdcc06fc8e8675` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:30:07.662224 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020337c-ff70-4f84-8870-207dba3e33fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef69c081a67644c7aecdcc06fc8e8675)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:30:07.664256 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020337c-ff70-4f84-8870-207dba3e33fe admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:30:07.712976 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c020337c-ff70-4f84-8870-207dba3e33fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef69c081a67644c7aecdcc06fc8e8675', 'request_id': 'req-c020337c-ff70-4f84-8870-207dba3e33fe'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:30:07.722770 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c020337c-ff70-4f84-8870-207dba3e33fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef69c081a67644c7aecdcc06fc8e8675', 'request_id': 'req-c020337c-ff70-4f84-8870-207dba3e33fe'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:30:07.729356 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c020337c-ff70-4f84-8870-207dba3e33fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef69c081a67644c7aecdcc06fc8e8675', 'request_id': 'req-c020337c-ff70-4f84-8870-207dba3e33fe'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:30:07.740525 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c020337c-ff70-4f84-8870-207dba3e33fe admin admin] The token cache is being invalidate because project ef69c081a67644c7aecdcc06fc8e8675 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:30:07.740812 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c020337c-ff70-4f84-8870-207dba3e33fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:30:07.741455 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c020337c-ff70-4f84-8870-207dba3e33fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:30:07.743064 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1438/2903] 10.222.0.27 () {58 vars in 1232 bytes} [Mon Jun 27 21:30:07 2022] DELETE /identity/v3/projects/ef69c081a67644c7aecdcc06fc8e8675 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:30:17.403974 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=32,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:30:17.737650 dv-cinder-847733-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:35:20.132618 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4ce9b-9684-413d-9129-587ed0450c4c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:35:20.134300 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-daa4ce9b-9684-413d-9129-587ed0450c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:35:20.210485 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4ce9b-9684-413d-9129-587ed0450c4c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:35:20.270753 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5365f01-7a5b-45a6-8b8d-62ad04a217c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5fa9b0b0314ee9bb7e16b8464307b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc29b021d1646819137c89e93f33601', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:35:20.271422 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5365f01-7a5b-45a6-8b8d-62ad04a217c0 service placement] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:35:20.271744 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5365f01-7a5b-45a6-8b8d-62ad04a217c0 service placement] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:35:20.272023 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5365f01-7a5b-45a6-8b8d-62ad04a217c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:35:20.273816 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c5365f01-7a5b-45a6-8b8d-62ad04a217c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:35:20.285620 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5365f01-7a5b-45a6-8b8d-62ad04a217c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:35:20.287248 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5365f01-7a5b-45a6-8b8d-62ad04a217c0 service placement] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:35:20.297887 dv-cinder-847733-1-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1466/2904] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Jun 27 21:35:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)